Solved: Missing JDK - Error with Ant (Java build tool)

I ran into 2 errors while trying to build ZXing mainly due to the fact that I only installed the Java 6 Open JRE but I was missing the JDK (looked at /usr/lib/jvm/java-6-openjdk).


Unable to locate tools.jar. Expected to find it in /usr/lib/jvm/java-6-openjdk/lib/tools.jar

...... /zxing-2.0/javase/build.xml:41: Unable to find a javac compiler;

com.sun.tools.javac.Main is not on the classpath.

Perhaps JAVA_HOME does not point to the JDK.

It is currently set to "/usr/lib/jvm/java-6-openjdk/jre"

Problem solved by installing the JDK from Ubuntu Software Centre.

You can also use: sudo apt-get install openjdk-6-jdk

Platform: Ubuntu 11.10

As a side note, I find it very useful to document all the major installations done on this blog mainly to aid my fallible memory.

Error: Piggyback SSL on Heroku - jetty.ssl.keypassword

I was using PiggyBack SSL (Heroku seems to have automatically enabled it for all apps recently) and I came across this error.. 

"jetty.ssl.keypassword "

Background: I am running a Clojure app (Noir) on heroku for testing. The app starts fine but the error message about the ssl keypassword made the app crash when I make a GET request to a https address under heroku.

Fix: I removed the ssl-port option for jetty options. It does not seem like it's needed.

However, if you have your own SSL certificate (not using PiggyBack), then you should set your own.

(def prod-settings 

  {:mode                  mode

   :ns                    'helen

   :session-cookie-attrs  {:max-age 3600

                           :secure  false}

   :jetty-options         {:port      (Integer. (get (System/getenv) "PORT" "8080"))}})

;; Removed ssl-port because heroku keeps requiring ssl password

;; Heroku's ssl-port is 443, no need to supply keystore or ssl password if using PiggyBack SSL

Collection naming problems with MongoDB - Do not include hyphens

Problem: Wrongly named collections in MongoDB will cause exceptions

Why the problem occurs:

I found most of my tips from this article.

http://blog.shlomoid.com/2011/08/how-to-fix-erroneously-named-mongodb.html

Collection names should begin with letters or an underscore and may include numbers; $ is reserved. Collections can be organized in namespaces; these are named groups of collections defined using a dot notation.

You can create collections with names that include hyphens. As explained in the article above, the db.runCommand for creating a collection can look like:

db.runCommand({"create":"test-col"});

However, since the javascript console will interpret collection names with hyphens in them as subtracting letters, you will get errors.

Solution:
https://jira.mongodb.org/browse/PHP-282

You need to be in to your admin db (and you might need to login first, might not for local)

> use db1
> db.auth(user, pass)
> use admin
> db.runCommand ({renameCollection: "db1.test-col", to: "db1.testCol"})

If that does not work, you might need to repair your database using mongod.
http://www.mongodb.org/display/DOCS/Durability+and+Repair

For my Clojure web app, I tried to keep to a naming convention for my data (using hyphens for spaces) and my functions (using underscore for spaces) but I got a gotcha with Javascript (I think this will happen when using ClojureScript too). 

Setting up self-signed SSL certificates for your Jetty instance (experiments with Noir and Clojure)

Hi everyone,
  Recently, Heroku has included support for Clojure (fantastic!) and I have been testing it out with Noir. There are several good tutorials on how to get started with Noir (a route-based website development framework with Clojure) and MongoDB (congomongo) on Heroku: http://thecomputersarewinning.com/post/clojure-heroku-noir-mongo http://devcenter.heroku.com/articles/clojure
  Interestingly, Heroku has also enabled free SSL (PiggyBack SSL), which is useful for testing apps. Free hosting, database and SSL, why wait? =)

  This is to document my steps in setting up a self-signed SSL certificate on your own machine for development purposes (Jetty, not Apache). I thought I would be done in 2 hours.. but I ended up taking 2 days. Sigh.. hopefully, this will help someone else.

  There are 2 ways to do this. You can either use keytool or both keytool and openssl. I used openssl and keytool. In hindsight, keytool seems to be easier (thank you Brenton - http://formpluslogic.blogspot.com/2010/08/securing-clojure-web-applications-with.html) and less problematic than using openssl to create the necessary files, then using keytool to import the files.

  FYI, I'm using Windows 7 (you will need to change the commands and paths accordingly), Noir 1.2.2, CongoMongo 0.1.5-SNAPSHOT, Clojure 1.2.1. The 2 programs that you need are OpenSSL ("../GnuWin32/bin/openssl.exe") and Keytool (which you can find in "../Java/jre6/bin/keytool.exe").
OpenSSL - http://devcenter.heroku.com/articles/csr (download the proper version for your system)

  As an overview, you will need to do the following steps:
Install openssl
Use OpenSSL ("../GnuWin32/bin/openssl.exe") to:

• generate a private site key (site.key)
• strip the password from site.key for automatic loading when uploaded to a server
• generate a self-signed signing request (site.csr) (might need it for Heroku)
• generate a self-signed certificate (sitex509.crt - in x509 format for loading into the keystore)
• combine the self-signed certificate (sitex509.crt) and site key (site.key) and export it in pkcs12 format (site.pkcs12)

Use keytool ("../Java/bin/keytool.exe") to:

• import the file site.pkcs12 into the keystore (sitepkcs12.keystore)

I tripped up along the way for many of these steps, so I will include the error messages too for reference.

Install the appropriate version of openssl for your operating system.

OpenSSL
You might need to enter some pass-phrases or passwords. I suggest that you write them down these up, just in case.


Generate a private site key (site.key)
$ openssl genrsa -des3 -out site.key 2048


Make a copy of site.key and strip the password, so that it can be auto-loaded when uploading to a server
$ copy site.key site.orig.key
$ openssl rsa -in site.orig.key -out site.key


Generate a self-signed signing request (site.csr) (might need it for Heroku)
Error: could not find openssl.cnf in the config
You will need to find a copy of the openssl.cnf. I used the one that was in "GnuWin32\share\openssl.cnf". If you are using Linux or OSX, you should be able to find your version. Btw, the version of openssl.cnf that I used was dated 2005 and it still seems to work.
$ openssl req -new -key site.key -out site.csr -config "..\GnuWin32\share\openssl.cnf"
You will need to key in the information requested (please refer to http://devcenter.heroku.com/articles/csr for an explanation). Please fill the proper info for "Common Name", it should be the secure domain or sub-domain. I suggest that you save this info, as you will need to enter the exact same info for the certificate. You can also skip entering any info for the "extra" attributes.
For example,


  Country Name (2 letter code) [AU]:SG
  State or Province Name (full name) [Some-State]:SG
  Locality Name (eg, city) []:Singapore
  Organization Name (eg, company) [Internet Widgits Pty Ltd]: myapp
  Organizational Unit Name (eg, section) []:
  Common Name (eg, YOUR name) []:localhost
  Email Address []:


  Please enter the following 'extra' attributes
  to be sent with your certificate request
  A challenge password []:
  An optional company name []:


Backup your site.csr.

Generate a self-signed certificate (sitex509.crt - in x509 format for loading into the keystore)
$ openssl req -new -x509 -key site.key -out sitex509.crt -config "..\GnuWin32\share\openssl.cnf"
Enter the same info as above.
Error: not in x509 format..
The certificate needs to be in x509 format or keytool will not be able to import it into the keystore as it cannot recognize it.

Backup your sitex509.crt.

Combine the self-signed certificate (sitex509.crt) and site key (site.key) and export it in pkcs12 format (site.pkcs12)
$ openssl pkcs12 -inkey site.key -in sitex509.crt -export -out site.pkcs12

Backup your site.pkcs12.


Keytool
Copy the file site.pkcs12 to your "..\Java\jre6\bin\" directory


Make sure you have full control (write, read-access) to the Java directory
Error: I had an error initially as I could not write to the Java directory. Go to the folder settings and enable the permissions. For Windows 7, you can add "Everyone" to the users and set "Full Control" for "Everyone".


Import the file site.pkcs12 into the keystore (sitepkcs12.keystore)
$ keytool -importkeystore -srckeystore site.pkcs12 -srcstoretype PKCS12 -destkeystore sitepkcs12.keystore

Double-check the keystore.

$ keytool -list -v -keystore sitepkcs12.keystore

Backup your sitepkcs12.keystore.

Noir

Copy all the files (site.key, site.csr, sitex509.crt, sitepkcs12.keystore) to your Noir project directory ("../myapp/").
I think only the keystore file is needed.
Error: javax.ssl does not .. correspond .. cipher.
You need to convert both the key and the cert (in x509) to pkcs12 format and import them into the keystore. Then, place the keystore in your Noir project folder.

Change your jetty settings
You will need to pass the settings (jetty-options as a map) to Jetty (remember your password!). In server.clj, for example,

(defn -main [& m]
  (let [mode      (keyword (or (first m) :dev))
        port      (Integer. (get (System/getenv) "PORT" "8080"))
        ssl-port  (Integer. "443")]
    (def myappserver (server/start ssl-port
                       {:mode           mode
                        :jetty-options  {:port      port
                                         :ssl-port  ssl-port
                                         :join?     false
                                         :ssl?      true
                                         :keystore "sitepkcs12.keystore"                    
:key-password  "abcdef"}
                        :ns             'myapp
                        :session-cookie-attrs  {:max-age 3600
                                                :secure  true}}))
    (users/db-init)))


Apologies for the weird formatting, please adjust accordingly for your app.

Go to https://localhost:443/myapp and http://localhost:8080/myapp to test.

Ask yourself daily how you are making someone's life better

Today, I learnt a very important lesson about startups. After 1 year of hitting walls together and self-reflection, I have arrived at a definition of a startup (not the definitive one, just my personal understanding). The startup process is a daily journey of pain and joy trying to honestly answer the all-important question for your customers: "How are you making my life better today?"

Appreciating the stochastic nature of life, failures and success

I think it would be good if people realized how random the world is (at least, it seems random to me =) ).

Imagine if all the things you ever did in life could be plotted as a normal distribution. Failures would then be considered as only data points on the curve. Instead of being overly concerned with a single data point, we would  be more interested in the overall shape of the curve.

There are 2 properties that characterize the shape of the normal distribution, the mean μ and variance σ². Hence, we would start asking ourselves different questions. Are our attempts leading us closer to the mean? Are we increasing the peak of our mean? How many standard deviations are our attempts away from the mean?

We would accept that life is inherently random, and that there is always a slight possibility of failure, regardless of how much we prepare ourselves. We would not put so much pressure on ourselves to always get it right or to always succeed on every attempt. Moreover, we would understand that trying and failing is just merely part of a process of trying and failing enough times, while trying to move closer to our peak and improving our peak.

We would not sweat the small stuff. Instead, we would keep a close look at how the middle part of our curve looks like (the bulk of the curve within 3 standard deviations).

What would be the value as measured by the vertical axis? Would it be money? Happiness? "Quality" (as defined by Robert Persig)? Would there be more than 1 dimension measured? It would be difficult and pointless to compare.

We could rank the activities that we do daily and start dropping all those activities that are not worthwhile.

Similarly, we could plot the relationships that we have and focus on the ones that make us happy.

If we were to compare between the curves of our friends and our own, we would realize that there would be a large overlap. The large overlap enables us to relate to each other and the differences make us interesting to each other.

10,000 experiments to find the material for a light bulb. 10,000 hours of deliberate practice to become an expert. 10,000 attempts for a good approximation to a life's worth of normal distribution =)

Relativity among humans

The biological origin of the relativity bias
As the human mind is biological in origin, it is not naturally inclined to frame thoughts in absolute terms. Having no reference to an absolute definite scale, the human mind is forced to generate thoughts, pass judgments and form conclusions using relative references. This has wide-ranging implications for the behaviour of individuals, groups and societies.

Relativity among individuals
Relativity and sensation

Psychologist Cialdini's experiment with subjects reporting their sensation of water at room temperature is an example of relative bias affecting the rationalization of individuals. Cialdini instructed subjects to first place one hand in a bucket of warm water, then the other hand in a bucket of room temperature. The subjects reported that the water at room temperature felt cold. Similarly, he repeated the experiment, but changed the warm water to cold water. The subjects subsequently reported feeling that the water at room temperature felt warm. 

Relativity affecting how the weights we ascribe to recent and historical events
This simple experiment shows 2 aspects of human irrationality that we often fail to acknowledge - the tendency to ascribe value in a relative manner and the recency effect. The recency effect refers to the way we tend to weight events that happened recently more than events which happened in the past.

Why we treat wins and losses differently

The relativity bias also affects how we treat potential losses and wins. Prospect theory demonstrates our aversion to loss and also our risk-seeking behaviour. (http://en.wikipedia.org/wiki/Prospect_theory) Simply put, we would walk 20 minutes to save $1 on a $4 plate of noodles, while we would not walk to the next Apple store to compare prices for a $700 iPad 2. For losses, we can examine the experiment conducted by Richard Thaler, where students display significantly different behaviours in 2 cases (http://www.sjsu.edu/faculty/watkins/prospect.htm). The first group of students were told to assume that they had won $30 and vote for a coin flip to decided whether they would win or lose $9. 70% of students voted to toss the coin. Another group were then given $30 and offered a chance to flip a coin to decide whether they would receive $21 or $39. 43% of students in the second group chose the coin flip. Thus, when people have gained something, they are reluctant to lose what they have gained. When people have nothing, they tend to adopt risk-seeking behaviour.

Since Prospect Theory seeks to explain the anamolous behaviour through a S-shaped utility curve, I would like propose that relativity bias be used to explain the origin of the S-shape. Relativity affects us in 2 ways, thus shaping the curve. For the example of lunch and iPads, we unconsciously or consciously judge the savings relative to the absolute amount we are intending to spend. Relativity also affects us in an emotional sense through our perceived relative social status. Once we have gained something, we perceive ourselves to be ranked higher in terms of social status and we are reluctant to lose that increased rank.

Examining the concept of identity, we can also break down an individual's perceived identity in terms of how an individual visualizes himself or herself relative to the other people whom he or she interacts with. We assume different roles according to the time and group that we are in.

We can thus attempt to understand individual human fallacies such as the attitude towards sunk costs, gamblers displaying increasingly desperate behaviour and the failure to learn from history.

Relativity among groups
Envy driving economic exertions

What happens when the effect of the relativity bias is extended to groups? Many people would instinctively choose the option of winning $5,000 in a lottery while their neighbours gain nothing, than win $50,000 while all their neighbours win $100,000. This explains the general disquiet felt when someone gets a brand new sports car while the rest of their friends are driving normal sedans. This explains the rush to keep appearences up with the neighbours, office politics and the behaviour of relatives at family gatherings. The annual jockeying of positions is keenly felt by those who come from a Chinese family, although I believe this behaviour is present in most cultures. Hence, as income inequality increases, society will naturally experience greater unhappiness and disquiet until a great reset. Greed may not drive the economic progress as much as envy (http://falkenblog.blogspot.com/2010/03/why-envy-dominates-greed.html).

Fluctuations in the stock markets

If we allow ourselves to treat the stock market as an emergent system from the behaviour of individuals, the bias due to relativity becomes useful in explaining the inherent irrationality present in stock market movements. The Efficient Market Hypothesis is only roughly right, with the aberrant behaviour ("Mr Market") probably being explained by the relativity bias. Since relativity affects our decision-making processes with respect to time (recency), losses and wins, it is understandable why people cut their losses too late and why people sell their winners too early. It would be resisting typical hardwired human tendencies to not do so.

The relativity of morality and social norms

What is morality and social norms? If you merely hold yourself and others true to certain principles and commitments, you are merely being self-righteous. I will attempt to define morality as a common but not universal unexclusive dynamic intertwined commitments among a large porous group of people that emerges from the set of mutual commitments. Morality and social norms only become powerful as a set of mutually binding commitments relative to the individuals in a group or society. However, the boundaries of the group are porous, the dynamic set is largely common but not universally agreed upon and the way morality emerges cannot be controlled. Hence, it is logical to follow why morality becomes extremely subjective and divisive, as undeniably, there exists relative standards of commitments as perceived by each individual within the porous group. Regardless of such an argument, I do believe there exists universal human truths such as the "Golden Rule", which seems to be repeated across multiple monotheistic and multitheistic religions.

Possible counters against relativity

Are we able to counter this relativity bias? This relativity bias is not always bad. If we cast our future selves as always being relatively better than our current selves, this personal reflection using relative judgements is useful for self-improvement.

It is my own personal hope (warning: introduction of subjective human bias) that we are able to detect signs that we are being subjective and objectively correct our thought processes. Remarkably, Buddhism and Stoicism have devised practices that address this fallacy of relativity bias in human behaviour.

Both Buddhism and Stoicism seems to emphasize a clear recognition of desires or insatiables. Having recognized the presence of these desires, one cultivates a mindset that reflects upon these desires and devises practices to rationalize why those desires are unnecessary. One method used is the concept of "impermanence". Recognizing that most things life are transient, we are able to cast the desire or want as being a mere blip in any relevant timeline we choose. We can also choose to frame what we lack into what we already possess. Before getting angry, we can also observe the emotional state in which we are getting angry and find a way to cast the situation in a relative favourable light. Thus, only a person's imagination limits the number of alternative reference frames to switch to in a tricky situation.

On a deeper level, through daily cultivation, Buddhism and Stoicism works on removing the generation of thoughts that lead to such desires. Clearing the mist of such desires, one can then focus on true happiness.

Personally, I find this concept of relativity very useful. I find myself being extremely arrogrant most of the time. By shifting my relative frame of reference, I am attempting to find a way to reduce or remove this arrogance as way to introducing much-needed humility. I also tend to judge people extremely quickly and harshly. By shifting my frame of reference, I see the person completely and am able to appreciate the person's faults relative to their strengths. 

Clojure - Printing special characters " \ / { } # ' $

I was playing around with Clojure and I had a problem printing out the following special characters:

" \ / { } # ' $

I uploaded the solution here: https://github.com/ryanteo/clojure-utilities

You can define a string containing all these special characters, then you can access each character by its index as a string in Clojure is stored as a sequence.

Not sure why this works (no time to read the documentation), but it's my hack-around.

As a side note, I've attached this reference for printing special characters in other languages.

Reference:

http://langref.org/clojure+cpp+csharp+erlang+fsharp+groovy+haskell+java+ocaml+perl+php+python+ruby+scala/strings/declaring-strings/special-characters

A love story over a lifetime

Quoted from The Straits Times, Lee Wei Ling

He told her: "We have been together for most of our lives. You cannot leave me alone now. I will make your life worth living in spite of your physical handicap."

She replied: "That is a big promise."

Papa said: "Have I ever let you down?"

"Today is a public holiday in Singapore. Can I take a break from swimming."

Papa replied: "No, have a swim. You will feel better after that."

They had concluded that the one who died first would be the lucky one.

“For reasons of sentiment, I would like part of my ashes to be mixed with Mama's, and both her ashes and mine put side by side in the columbarium. We were joined in life and I would like our ashes to be joined after this life."

江蕙 - 家后

http://www.youtube.com/watch?v=8eJP4kwQJVs

Understanding Financial Leverage in a crazy world: UBS Trader losing USD$2 billion

http://www.guardian.co.uk/business/2011/sep/15/ubs-star-trader-arrest-career

Given that I don't understand or have studied finance, I shall try to understand what happened during this incident.

Recently, a UBS trader lost USD$2 billion while trading. Of course, there's a lot of outrage (heard that, seen that before..) about how such a sound financial institution could have such a lax financial control, but I think the whole system was designed from the start to incentivise such risk-taking behaviour.

http://www.thebureauinvestigates.com/2011/09/16/analysis-the-real-rogue-is-not-kweku-adoboli-but-delta-one/

"In simple terms Delta One Trading is buying or selling an investment instrument that you don’t own. Delta One desks trade financial derivatives, or investment vehicles that mirror closely the price of a real asset. Any position taken is offset or hedged."

You are trading virtual goods.

The trader was earning at least a comfortable mid-6 figures. Let's assume he earned around USD$300,000/year and his annual performance bonus is 24 months. His bonus would have amounted to USD$600,000. Of course, his bonus is based on performance. It is also apparent that his bonus dwarfs his salary by a lot. Therefore, he would definitely want to earn more by taking extreme risks than earning just his basic salary. Given that traders are regularly fired based on performance, he would be forced to chase profits. Most investment bankers are also only in it to make a quick buck and retire in 5 years, hence everyone is trying to earn as much as possible before burning out.

For him to lose USD$2 billion, he must also have had the chance of earning USD$2 billion. For a person earning ~USD$1 million/year, he enjoyed 1000x leverage. In order to employ that leverage successfully, he should have some combination of insider knowledge, intelligence, sense of timing, judgement and experience. Guess he did not.

I know things don't scale as simply, but would you let someone earning $1 buy something from you that costs $1000 on credit?

The other thing that puzzles me is the unknown identities of the parties who earned that USD$2 billion. Someone must have earned it. If someone did not earn it, then we are effectively creating more exotic virtual financial fluff and printing more useless paper money.

Back to a simpler and more rational life as an engineer in a startup.