Last edited by greenbreakfast, 2016-06-13 21:52:47

The main goal of this project is to demonstrate the Omega's capabilities in a practical webscraping application. We will be making a stock ticker using our Omega to retrieve prices and the OLED Expansion to display them.

We will be using python to do our webscraping as it provides us with a convenient set of tools to retrieve and parse web data as well as a shell script to print our stock information onto the OLED display.

# Overview

Tutorial Difficulty:

Beginner

Time Required:

15 minutes

Required Materials:

• The Omega

• The Expansion Dock

• OLED Expansions

Useful Experience:

• Using the OLED Expansion

• Using Python

• Using Shell Scripts

• Scheduling with Cron

# Background Info

Stock Tickers are used display the most recent selling price for a stock on the market. Historically, they have been the most fundamental tool used by "wall streeters" for tracking performance. There are plethora of free stock ticker services for your computer, phone, tablet ...etc. However, we will provide you with the back bone to make a highly customizable stock ticker on a dedicated mini-computer (Omega) and display(OLED Exp) in under 15 min!

# The Actual Process

## Step 1: Connect your Omega, Expansion Dock, and OLED Expansion

Firstly, connect your OMEGA and your OLED Expansion to your Expansion Dock. Refer to the picture below.

## Step 2: Setup Wifi

You will need to setup wifi so that we can install python and fetch stock data from google. Go ahead and enter the wifisetup command into your omega.

wifisetup

## Step 3: Update and Install Python

We will need to install the full version of Python so that we have access to some libraries necessary for webscraping. If you have already installed python skip to the next step. To do this simply enter,

opkg update
opkg install python

You may or may not need to clear some space on your Omega for the installation. If you are doing this right out of the box, you should be fine.

## Step 4: Create our Python and Shell scripts

Navigate to your "/" directory. Create your shell script file by executing the following command.

cat > stock.sh

You will be prompted to enter the contents of the file. Copy the following code into the file.


#!/bin/sh

oled-exp -c

echo $1 > /root/ticker.txt VAR=$(python ./stock_script.py)

oled-exp -i

oled-exp write \$VAR

You can save and exit the command by entering CTRL + D. Firstly, we add the the shell to our path so that we can run the file as an executeable. We then clear the display using oled-exp -c. We write the ticker of interest to the following file /root/ticker.txt. Next we run our python script and store the output to our string variable, VAR. Then we initialize the display and write the value of VAR, which is our ticker and price.

Now lets do the same with our python script.

cat > stock_script.py

#!/usr/bin/env python

import urllib

import json

myfile = open('/root/ticker.txt', 'r')

jfile=urllib.urlopen(site)

jsfile=jsfile.replace("\n","")

jsfile=jsfile.replace("/","")

jsfile=jsfile.replace("]","")

jsfile=jsfile.replace("[","")

ticker=a['t']

price=a['l_fix']

info=ticker+":"+price

print info


To start we added python to our environment so that it is executable from the command line or a shell script. Next we imported urllib and json libraries, these will allows us to webscrape information and parse the data for our desired information respectively. Next we open the file that stores our desired ticker and store it as a string variable. Then we append that to our URL. Then we will generate the URL and open the file from the web, urllib.urllopen(site). We will store it as a string using jfile.read(). Next we will clean it up using the replace functions so that it can be parsed by the json library. We use the json.load(jsfile) to create a dictionary. Then we retrieve the values of interest using the keys for ticker and price. We append the two and spit the output to our shell script.

## Step 5: Change Permissions

Make both files we just created executable by entering this:

chmod +x stock.sh stock_script.py 

## Step 6: Execute Script

Now let's see how Apple is doing.

./stock.sh AAPL

# Using the Project

Parts of this project can and are encouraged to be used in other projects.

## Going Further

To take the project one step further, let's create a stream by having the Omega run the same script once every minute. To do this we will use the cron tool. cron allows us to schedule jobs(commands/programs/script) for execution at a fixed time. First lets schedule our sript to run every minute. To do this, enter the command:

crontab -e

You will be taken to the vi-editor of the scheduling file which can be found in /etc/crontabs/root. Add the following line to your file and make sure to end the file with an empty line or #.

*/1 * * * * /stock.sh AAPL

You can save and quit the editor by entering ZZ.

Using this project as a template, we can go even further with webscraping projects. Maybe build a new headline streamer or an upto date weather tracker.

## Related Tutorials

If you enjoyed this project, you should check this out OLED-QR-Code-Generator.