IOT Info Display Architecture

The best way to learn something is using it. That is the reason I decided to make something actually work for me as a way to explore IBM Bluemix.

There are a lot of stuff in IBM Bluemix. The first thing that draw my attention is some of the API that provide real life information. My targets are AlchemyAPI that provide filtered news data, and Weather Company Data for IBM Bluemix. So, my project is to develop a system that can draw data that make sense to me from this 2 APIs, and present them nicely to me.

After 7 months of development, this project comes to a point that fulfill almost all my wishes. So now it is the time to slow down and do some documentation. In this post, I’ll outline the architecture of this project.

Before I jump into the technical detail. I need to describe what this project do. This project, I called it “IOT Information Display”. It is a device with a small screen that display weather data of the device position, and filtered news headlines. It should be suitable to sit on my desktop and always on. So I can see the information any time. There is an Android App that I can use to change the device position setting and the news filtering rule.

For weather information. It can display current temperature and weather status. On the same screen it shows the temperature forecast of the coming 24 hours, and the probability of rain in the coming 24 hours in a chart. For news, it simply display headline one by one. I can tab on button to make it send the detail and source URL of such news to me in a email. There is another that can make it read the news detail with the speaker.

Hardware

I used Raspberry Pi 3 as the core hardware, and I added the official 7” touchscreen for Raspberry Pi. I picked up an 3D printable enclosure from Thingiverse to support the screen, and I added an extra compartment to house the speakers. I picked a pair of 0.5W speakers. I used a USB sound module for sound output. It is not a must but it provides a sound input so it will allow me to add a microphone input to the project later on. Since the sound signal from the USB sound is too weak. I needed to add an amplifier breakout board to increase the sound volume.

Software

Backend

The entire backend of this project is built on IBM Bluemix. It starts from the IOT Platform Starter Boilerplate. Which provides a Node.js + Node Red platform application platform, and an API helps to manage IOT devices. Under this configuration, Node Red will use Cloudant non-SQL database to store all data. I used the same Cloudant instance to store my application data on the server side.

After this foundation was built, I added extra services that I need in Bluemix. AlchemyAPI and Weather Company Data API was the core. Also I need to add Watson Text to Speech API for news reading. As Bluemix does not provide a SMTP service, I used the Mail API service provided by mailgun.com for email sending. So the entire backend is on the cloud. And, if you play carefully, it can be totally free.

Frontend

On the client side, the OS for Raspberry Pi is Raspbian jessie. The client program is a Java Application. The Bluemix IOT comes with a Java API that allow programmers sending messages between the device and the Node.js platform easily. In the Java client application, I added a few library for some special feature. First I added JFreeChart library for chart rendering. And I need the Pi4j API for GPIO control, since I need to use GPIO to perform some digital control on the amplifier breakout. The worst problem I had during the frontend development was media playback, as I found there is big limitation on media format support in the Java Runtime for Raspberry Pi. Since the Watson Text to Speech API would return speech data in OGG format. It is difficult for a Java application to play a media file in this format on Raspberry Pi. My final solution is gstreamer. Gstreamer is a very flexible library for media processing and playback. It has a Java binding. So I can integrate it into my project.

Later on I shall share some interesting technical topics about this project. Stay tuned for my future posts.

How did I build an Android App with Gstreamer Support

Well, it is just a personal project of mind. I'm making an Android App that can play video sent from my Raspberry Pi. Since "VideoView" component that comes with Android doesn't work for me. I have to integrate GStreamer library to my App. I found it extremely difficult. First of all there were lots of changes on Android, Gstreamer, and the development tool since the existence of these stuff. So there are lots of Google result that do not work today. And those official documentation are already out of date. There are many small pieces you'll need to add by yourself. That's why I do my own howto document here. Cause I know I'll forget the detail in the future.

Get the Tools

First problem is the Android development tool. The Gstreamer-Android library is a NDK pre-built library. However as of today the official development tool for Android is Android Studio, which have no support on NDK. I know some Android developers had done great job to make NDK works on Android Studio, but I found that is out of my hand. So I choose the old tool "ADT for Eclipse". I just grabbed the latest version of Eclipse for Java Development (Luna release 2). And installed ADT plugin as instructed on here. This is just the beginning.

Second tool you'll need is the NDK. The latest version today is NDK-r10d. You just need to download it and extract it somewhere in you hard drive. And, once you started a new work space with Eclipse, remember you need to set the path of NDK in the Eclipse preference (Perference > Android > NDK).

Also you'll need the Gstreamer-Android library. You should get it from here. Note that the library is a pre-built native library so you need to choose the CPU architecture. The site do provide ARM, ARMv7 and x86 for you to choose from. To start with, get the ARM debug version first.

Get the Tutorial Source

There are two main stream versions of Gstreamer: 0.10 and 1.0. Actually the latest version as of today is 1.4.5. The old version was better documented, but you can't make 1.0 version works for you by just reading those old document. The development team had published 5 Gstreamer-Android tutorial programs. With detail document that provide section-by-section explanation. However, in the website for 1.0 you cannot find any tutorial code. Finally I found this, which is great work done by slomo who ported the 0.10 tutorial code to work under 1.0 API.  Just use git clone to copy the tutorial code to your own hard disk.

Build the Tutorial Projects

You can easily find the 5 Android Tutorial projects in the tutorial source. Use "New > Project" command. Then choose "New Android Project from existing code". Browse to one of the 5 folders for the tutorials. And I recommend to have Eclipse copy the files into your work space.

Here come all the tricky parts. There are still many small changes you need to make before you can make it work.

1. Define Environment String

Actually you only need to define GSTREAMER_ROOT_ANDROID. Define it in the project's preference. Under "C / C++ Build > Environment". It should points to the root of your Gstreamer-Android API library.

2. Add Some Magic in Android.mk

At this point if you try to build the project. Eclipse will hang at certain point. And you don't get any error or hint. Google bought me the solution. Do two things:

1. Add the line "ifdef BUILD_PROJECT" at the beginning of Android.mk under the jni folder in your project. And add the line "endif" at the bottom.
2. Open the project's preference. Under "C / C++ Build", un-check "Use default build command". And add "BUILD_PROJECT=1" to the build command

3. Define Path and Symbol

In the project's preference, go to "C/C++ General > Paths and Symbols". Here you need to add all directories that contain all the header files that your C code need. They would be come from the NDK (<ndk root>/platforms/android-xx/arch-arm/usr/include) and GStreamer (<gstreamer-android api root>/include/<library name>). Without this even you can build your project successfully, the project will not run in the simulator as Eclipse still find "error" in the C editor.

4. Define NULL Symbol

Even with all above, you will still find one type of fault in the C editor. The editor cannot resolve the symbol "NULL". I don't know why it is not in any included header files. I think maybe the reason is I'm doing this on Windows. So I added it by putting these 3 lines under the include statements of the C code:


#ifndef NULL
#define NULL   ((void *) 0)
#endif

After all these. Try to run your project as an Android Application. If everything is fine, you'll get a working tutorial app. 

It is really a pain to do so many Google and troubleshoot all the problems one by one. I'm still wondering why I can't find a single instruction page or forum thread that give me all solution. Hope this can help you.

What will the next generation Operating System look like?

A few days ago I had a chat with my friends.  The topic goes to the latest trend of operating system(OS) evolution.  Considering web base applications are getting more and more powerful.  Casual users would rely on more web based application in the future, which means they will need to install less offline applications in their own hard disk.  The development of small size, relatively low end laptop computers (or netbook) is a good indication of this phenomena.  With this trend, I have made my bold prediction on the future of OS we would see in 5 to 10 years time.  You may or may not disagree my point, all I want is starting a discussion.

Web Browser Everywhere

More than 10 years ago, when someone saw a first generation web browser, like Mosaic.  He/she might not imagine how many functions it and the Internet can provide.  Now, we send email, edit documents, watch video, and play games with just a web browser in our own computer.  We will see that web browser would be a more advanced "Shell" in our operating system (like "cmd" in Windows and ksh in Unix), since every OS need a shell process to run applications.  Even better, web browsers on different OS all agree a bunch of open standard like HTML and Javascript.  That means such architecture brings another level of "write once, run anywhere" to application developers.  As the web browsers continuously evolve, we will have even more powerful and stable web based application.

So will the future OS really just a web browser with nothing else?  User will really no need to install any software locally? No, but things will work in a different way.

All Applications are Web Based

You may love to edit your documents on Google Docs, but you may not like to put any copy of your document onto the Internet due to some kind of security reason.  Someday you may be able to install a local version of Google Docs in your own computer.  Your OS will come with an application server that supports most commonly used web application architecture like JavaEE, PHP, and maybe .NET.  Installing a new software in you computer would actually installing a new web application package to the application server in your OS.  So you will still use the application with a Web Browser.  With a very similar look and feel as you use it online, but in fact it only hook up with your local application server.  Applications that require a lot of runtime resource will be deployed in this way.

Internet as Storage

As the bandwidth to the Internet becomes cheaper.  We'll found that buying storage on the Internet would be more cost effective than buying local storage for our own computer.  When uploading and downloading files to Internet storage provider would be fast enough at some point in the future.  The file explorer in our OS will be able to access Internet storage provider seamlessly.  Users can use them just like a local drive.  So when users want to have more storage space.  They will not buy new local hard disk, but pay to the storage provider for extra space.  The storage provider, will bear the responsibility on the maintenance of the storage infrastructure, backup of the users' data, and keep the data as secure as possible.  So, there will be no point to keep your files in local hard disk unless it is very confidential.

For example, today if you buy a Eee PC from Asus.  They provide 2Gb of Internet storage space to the buyer.  The file explorer of Eee PC can mount the storage space as a drive.  So managing files on it will be just like the files are in local hard disk (provided that your Internet connection is fast enough).

Local Database for Application

Almost all complicated application need a database of some sort.  Since the future OS will still contain local web based applications.  They should also provide some kind of local database engine, like mySQL, or file based database like Derby.  When a user install a local application, the installation program will create a new database locally if necessary.

Conclusion
At this moment, I think the nearest thing I can find on the market is Linux.  Maybe in a few years time a new Linux distribution will have some of the features I predicted.  The new OS from Goggle would be another hope since such kind of OS will be a perfect match to their current service.  What do you think?

Consideration of running batch programs without Stored Procedures

A few days ago, I've joined a discussion with one of my company's customer about software application architecture. Once again, the good old debate about using stored procedure(SP) was brought onto the table.

The customer, which used stored procedure for database operation in her legacy systems quite intensively, seems giving in this time. They agreed that for online transactions, DB operations can be implemented in Java based Data Persistence Layer(DPL), like DAO or Entity EJB. However, for batch programs, we proposed Stored Procedures can still be use. The customers seems happy. After all, this design looks nature, low risk, and DB programmers hired by the customer won't lost their job.

Running batch programs with SP seems a very nature decision. Batch programs usually involve large amount of data processing in database. And usually the time window for a batch program to run is very limited. Many application systems can only allocate 8 hours for running dozens of batch job everyday. So performance is also a key issue. SP usually is the fastest answer for this type of requirement.

Actually, from my experience, SP may not always help. In my career I've saw a weekly batch program with thousands of SQL scripts took more than 24 hours to finish. The problem is, no matter how fast SP can be execute, resource available for SP programs is only CPU and memory of one DB server.

The believers of Object Oriented Programming continuously have been selling us to put all persistent logic in DPL. In spite of the performance issue, using SP to update data in a batch process still defeat the whole propose of using DPL. After some time, the old data integrity issue will still emerge, as if no DPL is being used. So, if you think the hybrid solution I proposed to my client is the ultimate answer, you are in fact fooling yourself.

So, I re-think again and again, how a batch program can be implement with Java while all requirements, mainly on performance can be met? I have some answer below. If you are OOP people, please read them and kindly share your through with me. If you are SP people, please let me know how can you maintenance a batch program written in SP that need more than 24 hours to run without database re-design.

Problem 1: Performance

As I mentioned above, performance is usually the biggest hurdle against implementing batch programs outside the database. Why this have to be slower? The answer is simple. In this way the system have to send the data from DB to another server thought the network. Transform the data to another structure (e.g. Java Objects). And write the result back to DB thought the network again. Comparing with using SP, all these are additional overhead. As we cannot completely avoid these overhead. There is one way to compensate these.

When you run a batch program with SP, you have to use the CPU in the DB server. No matter how expansive your server box is. All you can have are usually 2 or at most 4 CPUs. With this limited computation power, they have to perform all the operation including query, calculation, and update.

When you perform your batch program outside your DB server. You are actually moving your calculation to other CPUs. In this way, not only you can have more CPUs to do the job, as you can have different servers to do independent batch programs. Also, your DB server would have less work to do, so queries and updates will go faster. With proper design, I think the performance will at least comparable with the traditional SP approach.

Problem 2: Memory Management

Surprisingly, there's a factor that make batch process with program outside the DB difficult, or sometimes consider impossible. It is memory management.

It is very common for a DB contains many Gigabytes or even Terabytes of data. In SP, it is relatively easy to create a temporary data set to store transient data for calculation. DB usually can handle this without much difficulties. However, for a program outside DB, memory available are usually a few Gigabytes, or less than 2 Gb for a Java program. If your program have to perform sorting or grouping on a large amount of data. You may easily hit the Out of Memory error.

The answer to this problem, is saving the data in temporary files on the disk. For example, with Java, you may implement common Java data structure such as List or Map to serialize the content and save them in temp files. As such actions will be wrapped in the data structure class, data read/write from the temp files would be hide away from the business logic.

Conclusion

After the discussion above, I have to say migrating batch programs from SP to something outside DB require considerable effort, and it is not risk free. However, we should not under-estimate the benefit of isolating the data persistence logic in the long run. If your application have a complicated database scheme, data in the database would gradually turn into a chaos if you let programmers to update the data directly in their business logic, even only in batch programs. Then causing painful maintenance issues afterward.

HtmlUnit - Java Headless Browser

HtmlUnit

When to use inner class

This is a question from a friend of mind.

In a class, we use private fields and private methods to hide, or encapsulate logic from outside. However, sometimes, we need a full functioning class within another class, but the class inside should not in anyway accessible or even exist outside the bigger class. This is the situation we need a inner class.

For example, inside a watch, there are many gears. As a user of a watch, you never care how the gears work within the watch. You even don't care if a watch is work on gears or not (it maybe a electronic watch). In this case, gear could be a inner class in a watch.

Private inner class provides better shielding of complicated logic within a class. The most common case of using an inner class is implementing Listeners inside another class. You just use an anonymous inner class that implements a specific Listener interface.

Although you can also define an inner class as a public class, but usually I do not recommend programmers to do this. Because you can always extract public inner class as an ordinary public class. I think it make no sense to use a public inner class unless you want to do something dirty.

Rethink Java and Object Oriented Software Engineering

I worked as a software developer for over 10 years. My job gives me many opportunity to lead and mentoring developers with very few or no experience on Object Oriented Software Design and Java programming. They usually ask me many questions. Some of these questions are very conceptional and not every experienced Java programmers can answer them well. If you want to find an answer in books, most books are too practical so they can just tell you how-to but seldom tell you why.


Most of them send me those questions with Instance Messenger . It is very difficult answer them with IM since it usually requires hundred of words to explain my idea. It is also a nightmare to send source code with IM. That's why I decided to answer those Java and OO Design FAQ with my blog.


As the very first chapter of the series. I'm going to recap the brief history of Java, and, as one of the most accepted language for OO Software Development, some characteristic of the language that make it so popular. If you have been using Java for a long time. You may not aware of these features have been helping you a lot. If you are programmer that have only use other computer languages like VB or PL/SQL. Please read on and you'll know why so many programmers shifted to use Java for software development.


Before the debut of Java, OO software engineering had been a hot topic. Like many new technologies, at first it was discussed in the academic circle. Then many research work was done in universities. There are quite a number of OO computer languages was born in 60s and 70s of last century. One of them was Modula 3, it have been widely used in OO software design classes in universities, but now it is widely forgetten except in university campus. Later on, Stroustrup, a Bell Labs worker, added OO feature to the C language, which was one of the most popular computer language in that age. C++ appeared in 1985, rode on the similarity with the C language, C++ gained much popularity in a decade. Finally, Microsoft also adapted C++ as a language for developing applications on Windows platform, which is the well known Visual C++.


As C++ jumped from the academic to the business world. People found that the OO feature is too difficult for junior programmers to learn. Althought it is very extensible like the C language. There is no API standardization under C++. Making C++ code not portable between different platform. There are also some disadvantage that are inherited from the C language.

If you have not used C++ before, it is hard to imagine what the problem is. C++ is like Java with following difference:

• Supports class multiple inheritance, you can have a class extends from many classes.
• Don't have Interface
  
• Can access memory directly with pointers, some how you must use a pointer in many cases when you use an object.
  
• No auto garbage collection (if you don't know what is GC, you are spoiled by Java)
  
• Cannot "Write once, run anywhere", not even "Write once, compile anywhere"

If you know Java, you should have a rough idea now.


In early 1990s, James Gosling (known as the father of Java, YES!!) from Sun Microsystem, invented the Java language. It looks like C++ a lot but there are also many improvement like:

• Simplified object inheritance
• No custom operator
  
• No pointers, so object is object everywhere in you program, like Visual Basic
  
• Auto garbage collection, no more memory leak issue
  
• Have its own standard API library, for many common operation like I/O, String manipulation, date manipulation, networking I/O and database access.
• Last but not least, the mighty "Write once, run anywhere".

As a computer language, Java is much easier to learn and use than C++. Over a decade, Java continuously evolved to catch up the growth of the industry and the demand of Java programmers. That's what brings Java to the current place in the industry.


So much for the history, now we have a good OO language. However, before Java get the big piece of pie in software development industry. There were many other tools exist like PL/SQL, Power Builder, and Visual Basic. All of them have been very popular. They are not OO. Why OO is so important? Why OO design make software development easier. Many books about OO give you the answer, but most of them are very difficult to understand. Now I try to make it easier with an example.


Imagine you go to a restaurant, you sit down and a waiter comes to you. You give him your order. He takes your order, writes it on a piece of paper. Then he goes to the cashier, leaves the paper there for recording. Then the waiter himself walks into the kitchen and starts to cook. After a while your dishes are ready and the waiter brings them to your table from the kitchen.


You finish your meal, you ask a waiter to give you your bill. The waiter goes to the cashier, searches for the paper that recorded all your orders. Then he calculates the amount your need to pay. Then he comes back to your table with your bill.


Consider this restaurant is a software system. All "waiters" (actually they are also cook and cashier) are programmed by procedure based programming language like C language or PL/SQL. In this kind of system every program must works from end to end. Every program must handle every task that lies on its path. It is very difficult to make changes to this kind of system. If you want to change this restaurant from serving French dishes to Chinese ones. You need to fire all "waiters" and re-employ a new team.


Usually, we won't see a restaurant operate in this way because our world is a world of objects. Waiters, cooks and cashiers are different party but work together. There is separation of duties, which means a waiter don't need to know how to cook, and a cook don't need to know how to calculate the bill. In OO design we call this "separation of concern", which is the most fundamental idea of OO Software Engineering. A software system should be build with a group of objects that are inter-related. This is what we call a "object model". All features that a OO computer language provides can help the developers to realize the object model easier.


If you understand this idea, the new question waiting for you is: "How to build a object model correctly?" This is not a easy question, thanks to many gurus who do a lot of work of OO design. Now we have some loose guide lines called "Design Pattern". Those are answers to many common problems that we'll face in OO design. I'll talk more about them in my later articles.