Antoine Aubry

Reinventing the wheel since 1990

20. September 2011 12:42
by aaubry
0 Comments

MBNet for Android

20. September 2011 12:42 by aaubry | 0 Comments

Notice: Due to a redesign of the MBNet site, this application no longer works. I hope to update it someday. In the meantime, don't bother downloading it.

The MBNet service

Those living in Portugal will be familiar with the MBNet service. It is a service that allows to use a Portuguese debit card to make payments in sites that accept credit cards only. In practice, you login with a user name and password, enter a maximum amount, and it generates a virtual credit card number that can be used for a single payment for up to the amount you specified. This service is very useful and I use it for most of the payments I make on the Internet.

Sometimes I need to make payments by credit card and the only trustworthy Internet access available is my Android phone. That is a problem because the MBNet site is not designed for small touch screens. It has a virtual keyboard that is a mess to use on a touch device. Not satisfied with this situation, I decided to create an Android application that could be used as a front-end for the Web site. The result can be seen in the following pictures.
MBNet1 MBNet2

Downloading the application

Warning: When you download applications from unknown sources, you are exposing yourself to a risk. Before using this application, think about the following risks:

  • Maybe this application sends your user name and password to some random hacker?
  • Even if you trust me, maybe the application has a bug that could expose your password?
  • This site might have been compromised and the application has been replaced with spyware?

In general, instead of downloading the application, you should download the source code, review it and then compile it yourself. In any case, I will not be held responsible if something happens to you after installing it.

If you are still interested, you can download the MBNet application here.

Implementation details

Since MBNet does not provide a publicly accessible API, which is understandable due to the nature of the service, I had to resort to Web scraping to implement the application. Web scraping consists of having a program interact with a Web site as if a real user was accessing it. This technique has some drawbacks. One of them is that even minor changes to the site can cause the application to break. Since there's no other way to do it, we will have to live with these potential problems.

A web scraper can be implemented in various ways. Some of them include:

  • performing HTTP requests to the site;
  • embedding a Web browser in the application.

After trying both techniques, I found that the Web browser solution was simpler because a Web browser already provides cookie management and javascript evaluation. Cookie handling is handy because we don't have to care about them while sending requests. Javascript was necessary because this Web site relies on client-side script to protect the user name and password of the user.

The Android platform provides the WebView control to display HTML pages. Opening a Web page is as simple as invoking loadUrl on the WebView:

webview.loadUrl("https://www.mbnet.pt/mbnetAut.html");

By default, javascript is disabled. We need to enable it using the following code:

webview.getSettings().setJavaScriptEnabled(true);

Interacting with the Web page

In order to do something useful, we need to be able to interact programmatically with the page. We will want to fill fields, submit forms and retrieve text. Usually, Web browser controls provide some way to manipulate the DOM and to invoke Javascript code on the page. WebView provides none of these. It allows to register objects that can be invoked from Javascript, but not the other way around. That is a problem because the pages that we are loading know nothing about our objects, therefore they won't invoke their methods.

Fortunately, it is possible to use a trick to work around the limitations of the WebView. If we tell the WebView to navigate to a javascript: url, the browser will execute the code after the semicolon in the context of the current page. This is how bookmarklets are implemented. This allows to do the following:

webview.loadUrl("javascript:eval('document.write(\"hello world\")')");

However, if we want to execute non-trivial code, this approach becomes hard to manage. To fix that I registered a javascript interface with a method that returns the actual script to execute, and the navigate code simply invokes that method and evaluates the resulting script. The following snippet illustrates that method.

webview.addJavascriptInterface(new Object() {
	public String getScript() {
       		// The code can be obtained from a resource or whatever
       		return "document.write('hello world');";
       	}
}, "itf");

webviewloadUrl("javascript:eval('' + itf.getScript())");

Conclusion

After finding out how to communicate with the page inside the WebView, it proved a viable solution for implementing Web scraping. I encourage you to read the source code for more details.

License

This program, along with its source code, is free software. You can use it under the terms of the GNU GPL license. For more information please consult the license.txt file inside the source distribution.

blog comments powered by Disqus