Use the ElasticMQ cloud service to create a scale-able and highly responsive application.
We are going to deploy an app which receives twitter messages and processes them in the
same thread to do a sentiment analysis on products. (instead of products we are using country
names here to get more messages). You will notice that the processing gets stalled intermittently.
Use the ElasticMQ service to off load the processing to the “worker” app. You will notice
that there is no stalling anymore. If the number of twitter messages are overwhelming, you can
create multiple instances of the “worker” app.
To introduce Elastic MQ and demonstrate how they can be incorporated into your projects we will use a node.js application which processes a stream of data from Twitter. In our sample scenario, the messages arriving from Twitter are used in place of messages being submitted by web users.
In the sample application, when messages are received from Twitter they are processed by a function running in the same thread that received the message. The processing performs some basic analysis of the data to simulate the sort of backend processing your applications might do.
The complete source code along with detail instructions can be found at https://github.com/ibmmessaging/
Demo of the code is available on youtube https://www.youtube.com/watch?v=nJUsinEQxE
This sample can be executed on-prem as well using IBM MQ Light available at
For this lab, we are going to focus on BlueMix.
This lab assumes that you have a few common tools already installed. These pre-reqs and
where you can find them are listed below.
1. git command line tool
There are required to download the source code for the lab. Installation instructions can be
found here http://git-scm.com/downloads
2. Cloud foundry command line tools
Required to push code into BlueMix. The installation instructions can be found here
(Scroll down for the stable installers section for your OS instructions)
3. IBM Codename:BlueMix account
To push code into the BlueMix PaaS you need to sign up to the BlueMix beta with an account.
You can sign up here https://ace.ng.bluemix.net
4. Twitter Developer API keys
You can obtain these from https://apps.twitter.com
1. Extract the source files from git by using the command
git clone https://github.com/ibm-messaging/mqlight-sentiment-sample
(or download the zip file from https://github.com/ibm-messaging/mqlight-sentiment-sample)
This should extract the 4 projects that we will be using into the current directory.
2. Update the twitterkey.json files in each directory with your twitter application keys.
Getting application keys -> https://apps.twitter.com
- Set consumer_key in the twitterkey.json file to the API Key string in your browser
- Set consumer_secret in the twitterkey.json file to the API Secret string in your browser
- Set access_token_key in the twitterkey.json file to the Access Token string in your browser
- Set access_token_secret in the twitterkey.json file to the Access Token Secret string in your browser
Connect to BlueMix
1. Link to BlueMix
cf api https://api.ng.bluemix.net
2. Login to BlueMix (you need to provide your BlueMix id and password)
Deploy the app without ElasticMQ
1. Navigate to webapp-noworker.nodejs directory and deploy the app. In this example BenNoWorker is the domain and NoWorker is the name we want BlueMix to call the app. You will need to choose your own names for your app.
<code> cf push -c "node app.js" -n BenNoWorker NoWorker </code>
2. The NoWorker app is now deployed on BlueMix. You can access it at your equivalent of:
[Notice that there is a time interval of approx. 1 sec between two tweets]
3. Stop the app
cf stop NoWorker
Deploy the app with ElasticMQ
1. Navigate to webapp-offload.nodejs directory and push the app with nostart option.
<code> cf push -c "node app.js" -n BenOffLoad --no-start OffLoad </code>
2. Create an instance of ElasticMQ service. Either using the BlueMix UI in the brower or with the cf command line. Give it a name. In this example it is called BenElasticMQ.
cf cs ElasticMQ Default BenElasticMQ
3. Bind the BenElasticMQ service with the OffLoad App. Either using the BlueMix UI or the command line as follows:
cf bs OffLoad BenElasticMQ
4. Navigate to worker.nodejs directory and push the app with nostart and noroute options.
<code> cf push -c "node app.js" -n BenWorker --no-start --no-route Worker </code>
5. Bind the BenElasticMQ service with the Worker App
cf bs Worker BenElasticMQ
6. Start both the application – OffLoad and Worker
cf start Worker
cf start OffLoad
The app is now deployed on BlueMix. You can access it at your equivalent of: http://benoffload.ng.bluemix.net
[Notice that there is a time interval of only approx. 200 ms between two tweets]
Scale the application
1. Create more worker instances using the BlueMix UI or this command, where 3 is the total number of worker instances you want BlueMix to deploy – so in this case 2 more instances in addition to the existing one.
cf scale Worker -i 3