Hello and Welcome.
In this Lab we’re going to use Bluemix to create an application that monitors tweets in Twitter, determines the mood of those tweets, and then sends us an SMS message with the ones that are positive.
Seriously this will be easy.
Let’s get started.
1. Quick Guide to Sentiment Analytics
We’re going to get a score from this analytics – a value that represents the mood or sentiment the tweet indicates. If the vibes from the tweet is positive – e.g. “I love cats “This is a really awesome cat” the value we get from the sentiment analytics will be a positive number. If the tweets is neutral – e.g. “This is a cat” – the value will be zero or very low. If the feeling is negative – e.g. “Cat’s are stupid” – the value returned will be also.
We can then decide what we want to do with this analysis. We could create cool graphs or funky heatmaps showing the mood of tweets on a particular subject.
What we’re going to do in this lab is simply annotate the tweet with the outcome, score and the appropriate smiley face and send them to our debug window. Whenever a tweet is positive, we’re going to forward the annotated tweet via SMS text message to our mobile phone.
The form of sentiment analytics we’re using is quite basic. For more advanced Twitter analysis see Bluemix’s new IBM Insights for Twitter service.
- Collecting Tweets in Real-time
First let’s create an app that captures tweets as they are tweeted.
It’s going to be really easy.
- Log in to Bluemix.
- Click on the Catalog link at the top.
- Find the Node-Red Starter service in the Boilerplates section near the top and click on it.
Bluemix’s Node-Red Starter service will deploy a Node.js runtime, a Cloudant NoSQL database and a monitoring service all in one click.
- Give your Node-Red service a name and hit create.
Bluemix will now create and deploy your Node.js runtime, Cloudant NoSQL database in the cloud. That was easy!
- Now click on the big red button to open up the Node-Red editor.
Start by finding the Twitter input node. Input nodes have little grey terminals on the left. Output nodes have their terminals on the right.
- Create a Twitter input node by dragging it onto the canvas.
- Now double click on the Twitter input node you’ve created so we can configure it.
- Click on the pen next to the “Log in as” box.
- Click on the button to authenticate with Twitter. This will allow your Node-Red app to access Twitter using your Twitter ID.
- Authorise Node-Red.
- Now go back to the browser tab where Node-Red is running. You should see that your Twitter handle now appears in the dialog box.
- Click Add
- Finish configuring the Twitter input node, by choosing the hashtag or text you want to apply analytics to e.g. #cats
Ok, let’s check to see that Node-Red is able to pick up the tweets.
- Add a Debug node (Debug nodes are always output nodes).
- Now wire the terminal of the Twitter input node into your Debug node, like this…
Great. Now we need to deploy this app to Bluemix.
- Click on the Deploy button at the top right. It’s red because you’ve made new changes that haven’t been deployed yet.
Now make sure that your Debug console panel is displayed.
- Click on the debug panel, just below the Deploy button.
Great. Now it’s a waiting game. Hopefully you picked a popular hashtag… We need to wait until the next person tweets on that hashtag. When they do you’ll see the tweet appear in your debug window. The tweet is in a JSON format, with lots of useful meta-data.
You can hurry this along by tweeting on the hashtag yourself of course 😉
Fortunately people are always tweeting about #cats so I didn’t have to wait too long!
Ok, so Bluemix is getting the tweets. Now let’s analyse them!
- Applying Sentiment Analytics
Rather than write our own code to analyse the tweets we’re going to re-use someone else’s. Even better, this code is already supplied by Node-Red in the form of a handy node!
First delete the connection you made between the Twitter input node and the Debug node.
- Click on the connection and press the delete key on your keyboard.
- Now find the Sentiment Analytics node and create a node by dragging onto the canvas.
- Wire the Twitter input node into the Sentiment Analytics node.
Now let’s decide what to do with the output.
- Find the Switch node and add this to the canvas.
- Wire the Sentiment node into the Switch node.
- Double click on the Switch node to configure it.
First, we’ll pass the Switch node the value generated by the Sentiment Analytics. We’re passing a JSON object with all the twitter meta data. The Sentiment Analytics node adds a sentiment element to the JSON. The score is added as msg.sentiment.score.
- Enter “score” in the If msg. dialog box.
- Now add conditions as below, so that in each case the Switch node creates a separate output terminal.
- Use the +rule button to add additional conditions.
- Click OK when finished.
Ok. Now let’s write some actual code!
We’ll start with the top Switch node terminal, which we configured for cases when the msg.sentiment.score value is positive.
- Add a Function node to the canvas and wire the top output terminal of the Switch node into it.
- Double click on the Function node.
Let’s take the positive tweets and insert some text at the beginning indicating that it was positive and include the sentiment score so we can see just how positive it was.
Here’s the code I used.
- Copy and paste this into the Function.
- Make sure you only have one call to return msg.
msg.payload = "Positive Comment :-) Score: " + msg.sentiment.score + " " + msg.payload;
- Copy and paste your Function node a couple of times and edit the code for the neutral and negative outcomes.
For example for the neutral case:
msg.payload = "Neutral Comment :-| Score: " + msg.sentiment.score + " " + msg.payload;
And for the negative case:
msg.payload = "Negative Comment :-( Score: " + msg.sentiment.score + " " + msg.payload;
- Wire these new function nodes into the appropriate terminals of the Switch node – the Neutral function to the middle terminal, the Negative function to the bottom terminal.
- Now wire all the Function nodes into the Debug node so we can see the annotated tweets with the scores in the Debug console window.
- Don’t forget to Deploy!
Now we wait… and eventually you should see annotated tweets appearing.
Twitter rate limit
Occasionally you’ll see this error when working with Twitter. It means that the free developer APIs we use to talk to Twitter have pulled more tweets than Twitter allows us to do for free.
There are 2 ways to fix this. First, upgrade your developer API account to pay to access more tweets. Second, stop your Node-Red flow from reading from Twitter by deleting the node and re-deploying the flow. Then go and take a break, have a coffee, write a letter to a loved one, take an early lunch. When you come back later you’ll be able to carry on until you hit the limit again.
- Sending Positive Tweets via SMS
Finally, we’d like to notify ourselves whenever we find a positive tweet. Node-Red provides several built-in ways to do this. For example, we could:
- Re-tweet them back into Twitter with the Twitter Output node
- Email them with the Email Output node
- SMS them with the Twilio Output node
- Send them via HTTP to a Web application
- Store them in a variety of databases
For this lab, we’ll send them to ourselves via SMS Text message.
First, we’ll set up the SMS messaging node and check that it’s working.
Then we’ll wire the SMS messaging node into the flow we created.
Ok let’s go.
- First find the Twilio node and drag it on the canvas.
- Now double click on the Twilio node to configure it
- Under Service choose External Service
- Next to Twilio click on the pen icon.
Leave this screen open ready for us to set up a Twilio developer account and then we’ll paste the credentials into Node-Red here.
- Open a new browser tab or window.
- Navigate to https://www.twilio.com/sign-up/try-twilio
- Fill in the details to apply for a free developer account.
To activate your account, you’ll also need to supply your mobile number complete with the international dial code e.g. +254.
Twilio will send you an SMS with a code which you enter on the screen to finish activating your account.
Finally Twilio will assign you with a new virtual Twilio number.
Important – For this demo to work in Africa, make sure you are assigned a United States virtual number i.e. one that starts with +1 as the international dialling code.
If you are assigned a number elsewhere, follow these steps to select a US number otherwise the SMS service won’t work.
- Go to Numbers on the menu or https://www.twilio.com/user/account/phone-numbers/incoming
- Check whether or not your number starts with +1.
- If not, click on the number to get the full details.
- Click on the Release Number button
- Choose Get It Now to get another number
- Twilio may now generate a US number that starts in +1
- However if it does not click on Choose your own number
- Use the drop down to select United States. You may need to choose another country first to get it to select US numbers.
- Choose a US number starting with +1 and click Get Started.
Now follow these steps to make sure you allow Twilio to send texts to your country.
Important – Check your text message permissions
Another common issue, is that the Twilio service doesn’t active SMS messaging to all countries by default.
Check that you can send Texts to your country as follows:
- Go to your Account settings in Twilio or this address https://www.twilio.com/user/account/settings
- Click on the GEOGRAPHIC PERMISSIONS tab.
- Click on the Messaging tab
- Scroll down to your country or click on the top check box to select all countries.
- Make sure your country is checked.
- Go to your Account on the Twilio site or https://www.twilio.com/user/account/settings
- Scroll down to API Credentials
- Here you’ll find your AccountSID and AuthToken (click on the key icon to display this)
- Copy and paste these two values carefully into the corresponding spaces in the Twilio node in Node-Red.
- Copy and paste your Twilio (virtual) number into the From This is the number that texts will be sent from within Node-Red.
- Click OK.
- Now enter your own Mobile number in the SMS to This means that Twilio will send a text to your number.
Great. We’re ready to send texts. Before we wire this into the rest of our Twitter Sentiment Analytics app, let’s test it to make sure it’s working. Inject nodes are a great way to test nodes, by passing messages into any node they’re connected to.
- Add an Inject node to the canvas and wire this into the Twilio node.
By default the Inject Node sends a timestamp to the next node. You can edit the Inject Node to send something more interesting.
- Change the Payload type to String.
- Enter something interesting in the Payload box (not the Topic box).
- Deploy your flow.
- Now in order to inject this string into the Twilio node – and trigger it to send you an SMS, click on the small blue button on the left of the Inject node.
- Wait a moment – and hopefully your phone will buzz with a text from Bluemix.
- If you don’t get a text after several minutes, check the Debug window for error messages and ensure your Twilio credentials and numbers are correct and in the right places.
Ok great – so we’re sending SMS from Bluemix.
Now let’s wire this up so that whenever someone posts a positive Tweet, we get the tweet and the sentiment analytics texted to our phones.
- Wire the output of the Positive tweet function into the Twilio node.
Now the output from the Positive tweet function will be texted to your phone.
Now you can experiment with other things you can do with Bluemix and Node-Red with the Twitter sentiment analytics.
But first don’t forget to delete the connection into the Twilio node and Re-Deploy, otherwise you’ll continue to receive text messages forever and ever!