Load “My Maps” to MapView on Android

You can create maps with point of interests on Goolge Map easily, and then save them to “My Maps” or the new name “My Places” for future use. I created such a map that has Ramen restaurants in Bay Area where I live, so I can bring it up on my Android phone. Although I can view my customized maps by adding a “My Maps” layer to Google Map, I thought it can be useful if I have more control to the map. That’s why I have been playing with MapView and other related API’s, and would like to share my findings here. The source can be found at https://github.com/barryku/SpringCloud/tree/master/AndroidApp/HelloMapView. Here is a screenshot from the running app,

The application is based on HelloMapView sample you can find on http://seveloper.andorid.com with the following additional features,

  1. Parse KML output using Simple XML.
  2. Parse JSON output using Jackson JSON
  3. Add balloon tip on MapView

To get KML or JSON output from your maps or other public maps, just add &output=kml or &output=json at the end of the map URL.  For example, http://maps.google.com/maps/ms?msid=215285050751615203721.0004a6d0191fbec6372b7&msa=0&output=kml will give you the KML output of the map I use in this application. KML only gives me the name and coordinate of my place marks, so I added code to handle JSON output which has a lot more information that can be useful for some applications.

The JSON output unfortunately is not 100% JSON compliant, so I  had to use the following code to work around it,

private void setupJsonOverlays() throws Exception {
	GeoPoint point = null;
	InputStream is = getAssets().open("Ramen_in_Bay_Area.json");
	ObjectMapper mapper = new ObjectMapper();
    mapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
    mapper.configure(JsonParser.Feature.ALLOW_BACKSLASH_ESCAPING_ANY_CHARACTER, true);
	GoogleMap gmap = mapper.readValue(is, GoogleMap.class);

The balloon tip uses an approach derived from an excellent tip posted at http://deckjockey.blogspot.com/2010/01/android-baloon-display-on-map.html. I simply created a layout with a TextView and ImageView and use the following code to set it up and then display it whenever users taps on a place marker on the map.

private void setupBalloonLayout() {
	LayoutInflater layoutInflater = (LayoutInflater) getSystemService(LAYOUT_INFLATER_SERVICE);
	noteBalloon = (BalloonLayout) layoutInflater.inflate(R.layout.balloon, null);

	TextView title = (TextView) noteBalloon.findViewById(R.id.note_txt);
	title.setOnClickListener(new OnClickListener() {

		public void onClick(View v) {


	ImageView goButton = (ImageView) noteBalloon.findViewById(R.id.go_button);
	goButton.setOnClickListener(new OnClickListener() {

		public void onClick(View v) {


public void doTap(OverlayItem noteOverlay, String txt) {
	MapController mapController = mapView.getController();
	mapView.addView(noteBalloon, new MapView.LayoutParams(12*txt.getBytes().length,55,noteOverlay.getPoint(),MapView.LayoutParams.BOTTOM_CENTER));

I used getBytes().length to get the length of my text strings since some of them contain double-byte characters.

The logic of parsing KML and JSON uses Simple and Jackson JSON respectively which really simplifies the parsing. All I had to do is creating those mapping/binding classes accordingly albeit taking some try and error to get them working. The following is the code that works on KML (JSON part is similar),

private void setupKmlOverlays() throws Exception {
	GeoPoint point = null;
	InputStream is = getAssets().open("Ramen_in_Bay_Area.kml");
	Serializer serializer = new Persister();
	KmlRoot kml = serializer.read(KmlRoot.class, is);
	List<Placemark> markers = kml.getDocument().getPlacemarks();
	for (Placemark marker: markers) {
		point = getGeoPointFromCoordiate(marker.getCoordinates());
		Log.d(LOG_TAG, marker.getName());
		itemizedOverlay.addOverlay(new OverlayItem(point, marker.getName(), marker.getDescription()));

In order to center the map and display all the place marks, the following code was added,

private void adjustMapZoomCenter(){
	int minLat = Integer.MAX_VALUE;
	int maxLat = Integer.MIN_VALUE;
	int minLon = Integer.MAX_VALUE;
	int maxLon = Integer.MIN_VALUE;

	GeoPoint point = null;
	for (int i=0; i<itemizedOverlay.size(); i++) {
		point = itemizedOverlay.getItem(i).getPoint();
		int lat = point.getLatitudeE6();
		int lon = point.getLongitudeE6();

		maxLat = Math.max(lat, maxLat);
		minLat = Math.min(lat, minLat);
		maxLon = Math.max(lon, maxLon);
		minLon = Math.min(lon, minLon);

	MapController mapController = mapView.getController();
	mapController.zoomToSpan(Math.abs(maxLat - minLat), Math.abs(maxLon
			- minLon));
	mapController.animateTo(new GeoPoint((maxLat + minLat) / 2, (maxLon + minLon) / 2));
	//mapController.animateTo(itemizedOverlay.getCenter()); //may need to call populate() first
Posted in Android, Cloud, Google | Leave a comment

Share Web page to Instapaper on Android

I wasn’t able to save a Web page to my Instapaper when browsing on my Android phone the other day.  Therefore, I tried couple of apps on Android Market later.  Although they worked fine, I was intrigued to figure out how the “share page via” feature is done on Android.  Here’s the application I created and shared at https://github.com/barryku/SpringCloud/tree/master/AndroidApp/InstapaperApp.

It turned out to be quite easy to add your application to the list to handling “share page via” when a URL link on the addressed bar is clicked (long-click).  There are only two things you need in your Android app to do so,

  1. Add an intent-filter to handle text/plain using SEND action.
  2. Use intent.getExtras().get(Intent.EXTRA_TEXT) to retrieve URL.

The following are code snippets from my application,


    <action android:name="android.intent.action.SEND" />
    <category android:name="android.intent.category.DEFAULT" />
    <data android:mimeType="text/plain" />


public void onCreate(Bundle savedInstanceState) {

    SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(this);
    final String userName = prefs.getString("userName", null);
    db = new UrlLinkDataManager(this);

	Intent intent = getIntent();
	TextView msg = (TextView) findViewById(R.id.user_message);
	ListView lview = (ListView) findViewById(R.id.urlListView);
	isInvokedFromShareVia = (intent.getExtras() != null);
	if (userName != null) {
		if (isInvokedFromShareVia) {
			final String uri = (String) intent.getExtras().get(Intent.EXTRA_TEXT);
			Log.d(LOG_TAG, "uri:" + uri);
			ConnectivityManager cMgr = (ConnectivityManager) getSystemService(CONNECTIVITY_SERVICE);
			if (!cMgr.getActiveNetworkInfo().isAvailable()) {
			} else {
	} else {

The application uses AsyncTask to do a URL GET request to Instapaper. It can be adapted to do different actions on the URL received easily.  The GET URL for Instapaper is https://www.instapaper.com/api/add?username=%1$s&password=%2$s&url=%3$s.  More details on how to use Instapaper’s API can be found at http://www.instapaper.com/api/simple. The main logic of the AsnycTaks is as followed,

protected String doInBackground(String... params) {
	HttpParams httpParameters = new BasicHttpParams();
	HttpConnectionParams.setConnectionTimeout(httpParameters, 5000);
	HttpConnectionParams.setSoTimeout(httpParameters, 5000);
	DefaultHttpClient client = new DefaultHttpClient(httpParameters);
	String result = null;
	try {
		String requestUri = params[0];
		String userName = params[1];
		String password = params[2];
		linkUri = params[3];
		String url = String.format(requestUri, userName, password, linkUri);
		Log.d(LOG_TAG, "accessing:" + url);
		result =  client.execute(new HttpGet(url), new ResponseHandler<String>() {

			public String handleResponse(HttpResponse response)
					throws ClientProtocolException, IOException {
				HttpEntity entity = response.getEntity();
				String httpResult = null;
				try {
					httpResult = convertStreamToString(entity.getContent());
				} catch (IOException e) {
					httpResult = e.getMessage() + ".";
				return httpResult;

	} catch (Exception e) {
		result = e.getMessage() + ".";
		Log.e(LOG_TAG, e.getMessage());
	return result;
Posted in Android, Cloud | Leave a comment

Copy text to Android emulator

How do you enter long text, especially those non-meaningful passwords or codes, into your Android?  You can try a small application I developed and published on Android Market at https://market.android.com/details?id=com.barryku.com.qcopy.  It works fine for me on the phone, but not so much on emulators since sending email to an emulator is too much hassle during development.

Here’s an application I wrote recently to address this need.  The application which I named it Copy2Emulator will access a URL and copy the first line from the response to Android’s clipboard.   Although I wrote it for working with an emulator, it can be used with phones or other Android devices as well.  I also created a companion web application which allows me to change the content of the Web response quickly.  The web application can be deployed on any Java application servers such as Tomcat, and then use it with Copy2Emulator.  You can download them at the following locations,

The following are a few screenshots to show you how to use them,

1. Click menu, and enter the URL in Preferences to point to a place where you can easily change the Web response from the URL.

2. Use the Web application to change the text to be copied.

3. Click on the Copy button or menu to copy new text to clipboard. The first line of the Web response will be copied.

To install Copy2Emulator.apk, go to platform-tools folder of your installed SDK and use ADB to install.  For example, you can do the following on Ubuntu,

  • ./adb devices  (to find out your device name)
  • ./adb -s emulator-5554 install /tmp/Copy2Emulator.apk

To use the companion Web app on tomcat,

  • copy qcopy-1.0.war to webapps folder
  • access http://localhost:8080/qcopy-1.0/qcopy?text=textStringYoudLikeToCopy
  • point your Android URL to  (Android uses to access the host machine)

You can find source at https://github.com/barryku/SpringCloud/tree/master/AndroidApp/Copy2Emulator.

Posted in Android, Cloud | Leave a comment

Upload file in Android with Spring RestTemplate

In my previous post, I created an Android app working with Box.net. It uses Box’s REST APIs to do authentication, browsing, and downloading files on Box.  This post will talk about the upload feature I added to the application.

It turned out to be quite straightforward to do file upload with RestTemplate when it’s done through a multipart form post.  All you need are the following few lines of code to upload a file from your Android,

String fileToUpload = dir.getPath() + File.separator + fileName;
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
parts.add("file", new FileSystemResource(fileToUpload));
String response = rest.postForObject(uploadUri, parts, String.class, authToken, folderId);

Pretty sleek, right? You can easily bind the response to a Java object via the built-in JSON or XML converter if you are interested in the response after the postForObject() request. An example can be found at AuthActivity in my previous post. Although Spring Mobile for Android does add 1 to 2 MB to an Android application, it’s worthwhile for its power and convenience.

The upload feature will get list of files from the “download” folder using Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS).  The files will then be rendered in a ListView, and an AlertDialog will appear when the user clicks on any file in the list to confirm or cancel the download action.  Environment.getExternalStoragePublicDirector() is the recommended way to write public files that will be shared to other applications starting API 8 (Android 2.2). On my test, Environment.DIRECTORY_DOWNLOADS points to /mnt/sdcard/download.  The following is the code which takes care of the application logic I mentioned above,

final File dir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS);
String files[] = dir.list();
setListAdapter(new ArrayAdapter<String>(this, android.R.layout.simple_list_item_1, Arrays.asList(files)));
final ListView v = getListView();
v.setOnItemClickListener(new OnItemClickListener() {

	public void onItemClick(final AdapterView<?> parentView, View view, int position,
			long id) {
		final String fileName = (String) parentView.getItemAtPosition(position);
		new AlertDialog.Builder(parentView.getContext()).setTitle("Upload").setMessage("Upload " + fileName +"?")
			.setPositiveButton("OK", new DialogInterface.OnClickListener() {

				public void onClick(DialogInterface di, int arg1) {
					String fileToUpload = dir.getPath() + File.separator + fileName;
					MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
					parts.add("file", new FileSystemResource(fileToUpload));
					RestTemplate rest = RestUtil.getRestTemplate();
					String response = rest.postForObject(uploadUri, parts, String.class, authToken, folderId);
					Log.d(LOG_TAG, response);
					Intent browseIntent = new Intent(parentView.getContext(), BrowseActivity.class);
			}).setNegativeButton("Cancel", new DialogInterface.OnClickListener() {

				public void onClick(DialogInterface di, int arg1) {


The complete code can be found on GitHub at https://github.com/barryku/SpringCloud/tree/master/BoxApp/BoxNetApp.

Posted in Cloud | 1 Comment

Simple XML with Android and Box.net

Continued on my experiment with Box.net’s API using Spring MVC, I tried the same on Android.  The source code can be found at https://github.com/barryku/SpringCloud/tree/master/BoxApp/BoxNetApp.

It uses Spring’s RestTemplate for Android to access Box’s REST API’s.  The way it works is similar to what I have done in my previous work on Android with Spring.  The main difference is that Box doesn’t generate JSON output, so I have to parse their XML response instead.  Luckily Spring already has built-in XML (Simple XML Serialization) support, and it’s not that much different than working with JSON.  However, there’s a caveat in Android’s SDK that requires a little tweak to get it working.  To keep Android SDK small, it doesn’t include everything in javax.xml.*, as a result, you will need to run dex with –core-library.  This will package those missing javax.xml into your dex file.  The fix will become available in andorid-maven-plugin 3.0 (see http://code.google.com/p/maven-android-plugin/wiki/Changelog for details.)  Before that happens, you can try my workaround I posted on StackOverflow.

This Android application has two main activities (screens), AuthActivity and BrowseActivity.  AuthActivity is the entry point when you start the application which checks for authToken’s existence.  If none was found, it present the screen with a “Sing on” button to take the user to Box.net’s authentication page.

public void onCreate(Bundle savedInstanceState) {
    SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(this);

    Intent intent = getIntent();
    if (AUTH_INTENT_SCHEME.equals(intent.getScheme())) {
        Uri uri = intent.getData();
        Log.d(LOG_TAG, "processing auth callback: " + uri);
        String authToken = uri.getQueryParameter("auth_token");
        if (authToken != null) {
            SharedPreferences.Editor editor = prefs.edit();
            editor.putString(AUTH_TOKEN_KEY, authToken);
    Button login = (Button) findViewById(R.id.btnLogin);
    login.setOnClickListener(new OnClickListener() {

        public void onClick(View v) {
            String requestToken = getRequestToken();
            Log.d(LOG_TAG, requestToken);
            Intent authIntent = new Intent("android.intent.action.VIEW",
                    Uri.parse(getString(R.string.auth_uri) + requestToken));

        private String getRequestToken() {
            RestTemplate restTemplate = RestUtil.getRestTemplate();
            RequestToken resp = restTemplate.getForObject(getString(R.string.request_uri), RequestToken.class, getString(R.string.api_key));
            return resp.getRequestToken();

Once authenticated, the authToken is stored in SharedPreference for use till it expires.  BrowseActivity takes in the folder ID, and then does REST call, get_account_tree, to render result in a WebView.

protected void onCreate(Bundle savedInstanceState) {

    restUri = getString(R.string.rest_uri);
    apiKey = getString(R.string.api_key);

    SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(this);
    String folderId = prefs.getString(LAST_VIEWED_FOLDER, "0");
    authToken = prefs.getString(AUTH_TOKEN_KEY, "");

    Uri uri = this.getIntent().getData();
    if (uri != null) {
        folderId = uri.getPath().substring(1);
    Log.d(LOG_TAG, "loading folder from onCreate()");


BrowseActivity also has a menu which allows users to log out Box to clear the authToken

public boolean onOptionsItemSelected(MenuItem item) {
    switch (item.getItemId()) {
    case R.id.logout:
        RestTemplate rest = RestUtil.getRestTemplate();
        RestResponse response = rest.getForObject(getString(R.string.logout_uri), RestResponse.class, apiKey, authToken);
        Log.d(LOG_TAG, "log out with status:" + response.getStatus());
    return super.onOptionsItemSelected(item);

The following is a sample XML and its corresponding binding Java class,

<?xml version='1.0' encoding='UTF-8' ?>
        <folder id="0" name="" description="" user_id="12001548"
            shared="" shared_link="" permissions="dcouv" size="320908"
            file_count="" created="" updated="">
                <folder id="85546770" name="test1" description="" user_id="12001548"
                    shared="1" shared_link="https://www.box.net/shared/syquihyigb"
                    permissions="kcgtedopnsuv" size="247783" file_count="9" created="1305389671"
                <folder id="86461258" name="test2" description="" user_id="12001548"
                    shared="0" shared_link="" permissions="kcgtedopnsuv" size="6200"
                    file_count="3" created="1305752928" updated="1305942446">
                <file id="748234686" file_name="assetList.jsp" shared="0"
@Root(name="response", strict=false)
public class AccountTree {
    private String status;

    private List<FolderItem> folders;

    private List<FileItem> files;

    public List<FolderItem> getFolders() {
        return folders;
    public void setFolders(List<FolderItem> folders) {
        this.folders = folders;
    public List<FileItem> getFiles() {
        return files;
    public void setFiles(List<FileItem> files) {
        this.files = files;
    public void setStatus(String status) {
        this.status = status;
    public String getStatus() {
        return status;

A few screenshots of this application in action.

Posted in Android, Cloud | Tagged , , | 1 Comment

Using OpenBox API @ Box.net with Spring

Box.net is an enterprise service which offers businesses a document collaboration tool in the cloud.  It’s straightforward to get access to box.net’s API’s.  Just sign up with a free account if you don’t have one, and then create a OpenBox application at http://developer.box.net to obtain an API key for use in your applications.

I was trying out their API’s recently, and found it’s works well with Spring framework.  Therefore, I decided to create an application to see how easy it is to use Box.net in Java.  The example uses mainly Spring’s RestTemplate and Spring MVC.  Using RestTemplate with Box.net’s API isn’t different than using it with other restful Web Services, and I have also covered it’s usage a few times in my previous posts.  The source can be found on GitHub at https://github.com/barryku/SpringCloud/tree/master/BoxApp/BoxNet.  The excellent Restful Web Service support in Spring framework makes it easy to work with any cloud services in Java.  You may find there’s no need need of any Java API for using Box.net’s Restful Web Services if you already use Spring in your projects.

There’s a live demo I put on AWS Beanstalk at http://boxnet.elasticbeanstalk.com that you can try when it’s running. Click on the home icon will take you back to the root folder.

The more challenging part of the example, on the other hand, is handling Box.net’s authentication flow.  The OAuth like authentication flow is explained in detail at http://developers.box.net/w/page/12923915/ApiAuthentication.  I use Spring MVC to handle the 3-legged authentication flow, and the same mechanism can be easily adapted to handle other OAuth or OAuth-like authentications.  The authToken is stored as an HTTP session attribute to be reused for subsequent requests.  It doesn’t the following steps when the first time user accesses the application,

  1. store the request url in the HTTP session
  2. forward to “auth” controller to do user authentication as followed,
    1. use API key to get the request token(ticket) via RestTemplate
    2. redirect user to Box.net’s authentication page with the ticket
  3. handle callback from Box.net using “boxnet” controller
  4. save authToken to session
  5. redirect user back to the saved request url

step 1 – 2

public String getAssetListWithFolderId(@PathVariable("folderId") String folderId, Model model,
		@RequestHeader(value="output-format", defaultValue="none") String ouputFormat,
		 HttpSession session, HttpServletRequest req) {
	session.setAttribute(SESSION_ACTION, req.getPathInfo());
	List<Asset> assets = null;
	repositoryService.setAuthToken((String) session.getAttribute(SESSION_AUTH_TOKEN));
	try {
		log.debug("get folder: " + folderId);
		assets = repositoryService.getAssetList(folderId);
	} catch (UnauthenticatedException ae) {
		return "forward:" + getAuthUrl(ae);

	log.info("assets count: " + assets.size());
	model.addAttribute("homeUrl","/" + MVC_CONTEXT + "/assets");
	model.addAttribute("assets", assets);
	if ("json".equals(ouputFormat)) {
		return "jsonView";
	} else {
		return "assetList";

step 2A – 2B (auth controller)

public String doAuth(@RequestParam(value="requestUrl", required=false) String requestUrl,
		@RequestParam(value="authUrl", required=false) String authUrl) {
	log.debug("get request token via:" + requestUrl);
	Source response = restTemplate.getForObject(requestUrl, Source.class);
	String requestToken = xpathTemplate.evaluateAsString("//ticket", response);
	return "redirect:" + authUrl + requestToken;

step 3 – 5 (boxnet controller)

public String boxnetCallback(@RequestParam(value="auth_token") String authToken,
		HttpSession session) {
	session.setAttribute(SESSION_AUTH_TOKEN, authToken);
	String action = (String) session.getAttribute(SESSION_ACTION);
	log.info("after auth, forward to:" + action);
	return "forward:/" + MVC_CONTEXT + action;

In my application, I also used Spring’s XpathTemplate to parse the XML output from box.net’s restful API’s.  It will be nice if they can add JSON output later, so it can be consumed without doing XML parsing or JAXB.  The following is my implementation class which uses get_account_tree to retrieve contents of a given folder on box.net,

private static final String URI_GET_ACCOUNT_TREE = "{resetUrl}&action=get_account_tree&folder_id={folderId}&params[]=onelevel&params[]=nozip&auth_token={authToken}";

public List<Asset> getAssetList(String path)
			throws UnauthenticatedException {

		List<Asset> assets = new ArrayList<Asset>();

		//path will replace the vars in the restRequest URI template, i.e. {restUrl} and {folderId}
		Source response = restTemplate.getForObject(URI_GET_ACCOUNT_TREE, Source.class, restUrl, path, authToken);
		List<FileItem> files = xpathTemplate.evaluate("//files/file", response, new NodeMapper<FileItem>() {
			public FileItem mapNode(Node node, int nodeNum) throws DOMException {
				Element element = (Element) node;
				FileItem bfile = new FileItem();
				return bfile;


I configured my implementation class (a Spring bean) as followed,

    <bean id="xpathTemplate" class="org.springframework.xml.xpath.Jaxp13XPathTemplate"/>
    <bean id="restTemplate" class="org.springframework.web.client.RestTemplate"/>
    <bean id="boxnetRepositoryService" class="com.barryku.boxnet.service.impl.RepositoryBoxNetImpl">
		<property name="downloadUrl" value="${box.net.downloadUrl}"/>
		<property name="requestUrl" value="${box.net.requestUrl}${box.net.apiKey}"/>
		<property name="authUrl" value="${box.net.authUrl}"/>
		<property name="restUrl" value="${box.net.restUrl}${box.net.apiKey}"/>
		<property name="restTemplate" ref="restTemplate"/>
		<property name="xpathTemplate" ref="xpathTemplate"/>
Posted in Cloud | Tagged | 3 Comments

Working with RTM(Remember The Milk) API’s in Java

There isn’t much on using RTM API I could find on the net, so I spent some time to come up with my own. This experiment also helped an Android application I am developing. You can find out more details at about RTM API at http://www.rememberthemilk.com/services/api/.

In this post, I will show you how the authorization flow works in RTM.  The steps of using RTM API are very similar to other Rest API’s using oAuth.  The following are steps you will need to go through before calling your first RTM REST API, e.g. rtm.tasks.getLit,

  1. Get a frob using your API key and shared secret.
  2. Get an auth_token using frob
  3. Access REST API using auth_token

Although the steps are straightforward, it still took me a while to straighten out everything.  Hopefully the Java example below will save some people’s time on going through the same.  All the URLs have to be signed with the shared secret, so I added a helper method, getSignedUrl(), to do so in the code example below.  To apply for an API key, go to http://www.rememberthemilk.com/services/api/keys.rtm.  The auth_token you received in step 2 should be stored somewhere for reuse ideally, so you can do step 3 directly in the future.

package com.barryku.playground;

import java.math.BigInteger;
import java.net.URI;
import java.security.MessageDigest;
import java.util.List;
import java.util.Scanner;
import java.util.TreeMap;

import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URLEncodedUtils;

public class RtmApiFlow {
	private static final String SECURED_REST_ENDPOINT = "https://www.rememberthemilk.com/services/";
	private static final String UNSECURED_REST_ENDPOINT = "http://api.rememberthemilk.com/services/";
	private static final String REST_ENDPOINT = SECURED_REST_ENDPOINT;
	private static final String API_URL = REST_ENDPOINT + "rest/?format=json&api_key=";

	private static String apiKey;
	private static String sharedSecret;

	public static void main(String[] args) {

		Scanner in = new Scanner(System.in);
	    System.out.println("=== RTM's Auth Workflow ===");

	    //construct auth request URL using API Key and secret to obtain frob
	    System.out.print("Enter your API Key\n>>");
	    apiKey = in.nextLine();
	    System.out.print("Enter shared secret\n>>");
	    sharedSecret = in.nextLine();
	    String authRequestUrl = REST_ENDPOINT + "auth/?&perms=read&api_key=" + apiKey;
	    System.out.println("Copy the following auth request URL to your brwoser, and grant permission when prompted:");
	    System.out.print("Press enter to proceede....");

	    //get auth token using frob
	    System.out.print("Copy and paste the frob from the browser page\n>>");
	    String frob = in.nextLine();
	    String authTokenUrl = API_URL + apiKey + "&method=rtm.auth.getToken&frob=" + frob;
	    System.out.println("Copy the following auth TOKEN URL to your brwoser,");
	    System.out.print("Press enter to proceede....");

	    //get list of incomplete task in JSON format
	    System.out.print("Copy and paste the frob from the browser page\n>>");
	    String authToken = in.nextLine();
	    String taskListUrl = API_URL + apiKey + "&method=rtm.tasks.getList&filter=status:incomplete&auth_token=" + authToken;
	    System.out.println("Copy the following to your brwoser to get the list of incomplete tasks,");

	private static String getSignedUrl(String url) {
		String result = url;
		try {
			List<NameValuePair> params = URLEncodedUtils.parse(new URI(url), "utf8");
			TreeMap<String, String> sortedParams = new TreeMap<String, String>();
			for(NameValuePair param:params) {
				sortedParams.put(param.getName(), param.getValue());

			StringBuilder sb = new StringBuilder();
			for (String key : sortedParams.keySet()) {

			MessageDigest md5 = MessageDigest.getInstance("MD5");
			byte[] data = sb.toString().getBytes("utf8");
			md5.update(data, 0 , data.length);
			BigInteger i = new BigInteger(1,md5.digest());
			String md5String = String.format("%032x", i);
			result = url + "&api_sig=" + md5String;
		} catch (Exception e) {


		return result;
Posted in Cloud | Tagged , , , | Leave a comment

Moving files from S3 to Google Storage with Camel

Enterprise integration is hard and complicated without tools.  I used a ESB, CapeClear, for my last project, and was really amazed by its capability.  Integration with different systems in the cloud is a norm, but ESB or middleware are quite expensive and may be overkill for small projects.  A friend mentioned Apache Camel to me last week, and I found it has done a great job on realizing all the well known Enterprise Integration Patterns(EIPs).  It looks like it can be a very useful tool for integrating various services in the cloud.

I did an experiment with Camel to integrate Amazon S3 with Google Storage over the weekend.  You will find the source code at https://github.com/barryku/SpringCloud/tree/master/CamelApp.  You can run this program with Maven easily.  Just put in your Amazon and Google Storage’s credentials and bucket names to src/main/resources/META-INF/spring/spring.properties, and run mvn compile exec:java -Dexec.mainClass=com.barryku.camel.FileCopy.

The Java code below shows you how you can move files from S3 to Google Storage with just a few lines of code using Camel,

CamelContext context = (CamelContext) springContext.getBean("camelContext");
context.addRoutes(new RouteBuilder() {

	public void configure() throws Exception {
		from("s3file:///").beanRef("gsFileManager", "process");


There are a lot of ready to use components(endpoints) in Camel you can leverage with your projects, but nothing for S3 and Google Storage yet.  That’s why I spent last weekend to write my own implementation. S3 has been around for a while, so its Java API is mature and easy to use comparing to Google Storage’s. Even with the luck of finding an alpha version of google-api-java-client with Google Storage support early on, I still went through a lot of trouble getting it working with my program.  Anyway, let me show you the primary logic of working with both S3 and Google Storage.  The following is my s3file component implementation,

AmazonS3 s3 = new AmazonS3Client(
		new BasicAWSCredentials(apiKey, apiKeySecret));
boolean isRootFolder = path.equals("/");
ObjectListing objList = isRootFolder ? s3.listObjects(bucket) : s3.listObjects(bucket, path);

for (S3ObjectSummary summary:objList.getObjectSummaries()) {
	//ignore folders
	if(! summary.getKey().endsWith(FOLDER_SUFFIX)){
		S3Object obj = s3.getObject(
				new GetObjectRequest(bucket, summary.getKey()));
		logger.info("retrieving " + summary.getKey());
		FileOutputStream fout = new FileOutputStream(TEMP_FOLDER + (isRootFolder ? "/" + summary.getKey():
		InputStream in = obj.getObjectContent();
		byte[] buf = new byte[1024];
	    int len;
	    while ((len = in.read(buf)) > 0){
	      fout.write(buf, 0, len);

All S3 files are downloaded to a local temp folder when the Camel route component, s3file, is being set up. The code should be self-explanatory. On the other hand, working with GS is similar to working with a typical restful Web Service. However, it can be quite tricky to construct a restful GS request without the help of its Java client API. The following is the code that finally worked for me,

HttpTransport transport = GoogleTransport.create();
GoogleStorageAuthentication.authorize(transport, apiKey, apiKeySecret);

HttpRequest request = transport.buildPutRequest();
InputStreamContent isc = new InputStreamContent();
isc.inputStream = new ByteArrayInputStream(content);
isc.type = type;
request.content = isc;
request.url = new GenericUrl(url + bucket + "/" + URLEncoder.encode(fileName, "utf8"));
GoogleHeaders headers = (GoogleHeaders) request.headers;
headers.date = httpDateFormat.format(new Date());
try {
	HttpResponse response = request.execute();
	logger.info(fileName + " uploaded");
	//workaround for timeout issue after 3 consecutive connections
	//consume the response will ensure Appache's HTTP client close connection
} catch (HttpResponseException e) {
	logger.warn(getStreamContent(e.response.getContent()), e);

Although my implementation for s3file component and the gsFileManager bean works, it’s still kludgy. I plan to refactor later to make them more like real Camel components.  Here are a few tips that may save you time when seeing those 403 or other hard to interpret errors working with google-api-java-client.

  1. For Get requests for a given bucket, you must add a slash at the end of URL if you are using the format of http://bucketname.commondatastorage.googleapis.com/
  2. For Put requests, you must set type of your InputStreamContent.
  3. You will get MalformedHeaderValue error if you use PST, however, pst and GMT work just fine.
Posted in Amazon WS, Cloud, Google, SOA | Tagged , , | 2 Comments

Yahoo oAuth using Scribe

It’s very common in this Cloud age to have a need to connect different applications in the internet. This can form a complete solution without reinventing the wheels.  How to you get access to end user’s data if such need arises? oAuth is probably what you will encounter these days for getting data access permissions from end users. That’s the reason why I was playing with Yahoo’s oAuth this weekend to get myself familiar with oAuth.

I was trying to get the end user’s contact information from his/her Yahoo account.  Scribe seemed to be quite easy to work with, and I had already used it a bit with Spring Social a few weeks back.  It is indeed a nice library, however, there were still several things I had to iron out to get it working with what I was trying to do.

The follow are the steps I went through,

  1. Register a standard Web-based project https://developer.apps.yahoo.com/projects with read access to Contacts.
  2. Add a html file with Yahoo’s provided name to my domain’s public_html folder (this is similar to Google’s process when you register a domain for your applications)
  3. Download scribe-java (git clone git://github.com/fernandezpablo85/scribe-java.git)
  4. Build scribe with Maven (mvn clean test), so I can use its YahooExample in test folder.
  5. Run Maven java:exec to test YahooExample
    mvn exec:java -Dexec.mainClass=org.scribe.examples.YahooExample -Dexec.classpathScope=test

Yahoo uses GUID for many of their rest API’s, so I modified YahooExample to extract GUID from the raw response, and then get list of contacts for the give user who had granted the permission.

package org.scribe.examples;

import java.util.Scanner;

import org.scribe.builder.*;
import org.scribe.builder.api.*;
import org.scribe.model.*;
import org.scribe.oauth.*;

public class YahooExample
  private static final String PROTECTED_RESOURCE_URL = "http://social.yahooapis.com/v1/user/%1s/contacts?format=json";

  public static void main(String[] args)
    OAuthService service = new ServiceBuilder()
    Scanner in = new Scanner(System.in);

    System.out.println("=== Yahoo's OAuth Workflow by Barry===");

    // Obtain the Request Token
    System.out.println("Fetching the Request Token...");
    Token requestToken = service.getRequestToken();
    System.out.println("Got the Request Token!");

    System.out.println("Now open your browser to the following URL, and grant access when prompted." +
            "You will get a response page with a verification code since there's no callback specified. " +
            "Copy the verification code.");
    System.out.println("Paste the verification code here");
    Verifier verifier = new Verifier(in.nextLine());

    // Trade the Request Token and Verfier for the Access Token
    System.out.println("Trading the Request Token for an Access Token...");
    Token accessToken = service.getAccessToken(requestToken, verifier);
    System.out.println("Got the Access Token!");
    System.out.println("(if your curious it looks like this: " + accessToken + " )");

    // Now let's go and ask for a protected resource!
    System.out.println("Now we're going to access a protected resource...");
    OAuthRequest request = new OAuthRequest(Verb.GET,
            String.format(PROTECTED_RESOURCE_URL, getYahooGuid(accessToken.getRawResponse())));
    service.signRequest(accessToken, request);
    Response response = request.send();
    System.out.println("Got it! Lets see what we found...");
    System.out.printf("response code: %1s \n", response.getCode());

    System.out.println("Thats it man! Go and build something awesome with Scribe!");


private static final String YAHOO_GUID = "xoauth_yahoo_guid";
private static final int GUID_LENGTH = 26;
private static String getYahooGuid(String response) {
    String yahoo_guid = null;
    int yahoo_guid_location = response.indexOf(YAHOO_GUID);
    if ( yahoo_guid_location > 0) {
        yahoo_guid = response.substring(yahoo_guid_location + YAHOO_GUID.length() + 1,
                yahoo_guid_location + YAHOO_GUID.length() + GUID_LENGTH + 1);
    return yahoo_guid;

You will get a screen similar to the following when you run this class using mvn exec:java.

Posted in Cloud | Tagged , | 1 Comment

Cloud Mashup: SaaS (Twilio) + PaaS (Google AppEngine) + IaaS (Amazon WS)

This application is an effort to lower my phone bill by using Cloud for my SMS needs.  I text less than 50 messages a month, so I really didn’t want to shell out $10 a month for  my T-Mobile account or paying 20c per message.  Twilio has SMS service which costs me 2c per message, so I figured out a solution to send/receive text messages with my Twilio account.  The application I am talking about here solves half of the problem, and there’s another Android application I wrote which solves the other half.  I will post about that Android application when I get a chance later.

This is basically a Google AppEngine(GAE) application which receives Twilio’s request, and then forwards to SNS at Amazon WS(AWS).  I created a email subscription to the SNS topic which finally sends the text message to my Yahoo email.  The solution can certainly be simplified, but it is more interesting for me to be able to exercise Cloud services in different layers doing this way.   You will find all codes mentioned in this post at https://github.com/barryku/SpringCloud/tree/master/GaeApp.

You can have Twilio to trigger any HTTP request whenever a SMS message is received, so I created a Spring MVC application on GAE which will handle the callback.  The controller code is quite simple since the dependent service will be injected automatically by Srping.

	private AmazonSnsService amazonSns;

	@RequestMapping(value="/twilio", method = RequestMethod.POST)
	public void processCallback(@ModelAttribute("twilio") TwilioCallback msg) throws IOException  {
		String msgId = amazonSns.sendTwilioMessage(msg);
		log.info("SNS ID: " + msgId);

Amazon did a great job on AWS Java library, so using SNS is also quite straightfoward.

public class AmazonSnsService {

	private AmazonSNSClient sns;
	private String topicArn;

	public AmazonSnsService(AmazonSNSClient sns) {
		this.sns = sns;

	public String sendTwilioMessage(TwilioCallback msg) {
		PublishRequest request = new PublishRequest(topicArn, msg.getBody(), msg.getFrom());
		PublishResult result = sns.publish(request);
		return result.getMessageId();

	public void setTopicArn(String topicArn) {
		this.topicArn = topicArn;

	public String getTopicArn() {
		return topicArn;


The Spring configuration for AmazonSnsService can be found in mvc-servlet.xml. You will modify spring.properties to include your AWS information there.

<bean id="amazonSns" class="com.barryku.gae.service.AmazonSnsService">
		<bean class="com.amazonaws.services.sns.AmazonSNSClient">
				<bean class="com.amazonaws.auth.BasicAWSCredentials">
					<constructor-arg value="${amazon.accessKey}"/>
					<constructor-arg value="${amazon.accessSecret}"/>
	<property name="topicArn" value="${amazon.topicArn}"/>

There’s one caveat with using AWS on GAE though. GAE is running on a sandbox environment, so you will get a security error at Apache’s HttpClient when using AWS directly. I found a good workaround at https://github.com/apcj/aws-sdk-for-java-on-gae. It overwrites com.amazonaws.http.HttpClient using GAE’s UrlFetch which works beautifully with my application.

Posted in Amazon WS, Cloud | Tagged , , , , | 4 Comments