Note on using cron

Edited:1)From @sureshdr on twitter, My cronjob generate too many entry to log, so updated both on the server and the post with >/dev/null 2>&1 at the end of the job.

First time have to produce an app, and maintaining the server all by myself. Previously can work with the sysadmin, now the guys in sinar is pretty busy with a day job.

So first thing first, the thing with virtualenv is, running it in a cron job can be a pain(it is not that bad). Well, billwatcher is using buildout, so it uses less command, buildout have pretty much an isolated environment. So it look like this, more on why I don’t direct to /dev/null:

0 0 * * * /path/to/Malaysia-Bill-Watcher/bin/python /path/to/Malaysian-Bill-Watcher/billwatcher/loader.py >/dev/null 2>&1

Second thing is, the last deployment, I didn’t add a full path to the sqlite database. This is important, so that the data is written in the right place.

Third, because this is my first time running cron(don’t ask why). I finally found the log file to cron.

less /var/log/cron

Except, my cronjob is run at midnight, and a few cronjob run already. Then I found zcat, because the older cronjob is /var/log/cron.n.bz2. n is the number.

zcat /var/log/cron.0.bz2|less

I read it, make sure it is running, I’m happy, then I realized that I kinda polluting the email on my system which I don’t read often. So comes the MAILTO directive. Thus my final cron job look this way

MAILTO=”[email protected]
0 0 * * * /path/to/Malaysia-Bill-Watcher/bin/python /path/to/Malaysian-Bill-Watcher/billwatcher/loader.py >/dev/null 2>&1

And the newbie sysadmin live happily ever after… Until something else break, and there is a new project coming soon…

p.s running billwatcher at midnight is a bad idea, might be a better idea doing it in the morning..

Attempt in translating the billwatcher

there is request to see Malay support on the bill watcher for +Sinar Project which things begin to be interesting,

  • we going to scrape the page twice, one time for english version, one time for malay
  • now we need to have a way to save 2 translation of the same bill title
  • in which I don’t want to use a new table in db for that, gettext exist for a reason
  • at the same time, we need to save 2 different url for pdf for the language too,
  • now not just we need support the language, we need to load these into elasticsearch,
  • because we have to support 2 language, we have to load into elasticsearch twice, one for each language
  • while telling it, “it is really the same thing”
  • and did i tell you we need to translate the html template too? which the bottle own template language don’t have support for that

Most is relatively easy, the bill title is tougher, things can easily go out of hand, if using gettext, not really the issue of size, the compiled database is pretty ok.

The most interesting problem, how to index multiple language in elasticsearch while referring to the same bill.

Bill Watcher is LIVE!!!!

Thanks to kaeru, bill watcher is finally live, you can find it in

http://billwatcher.sinarproject.org/

What have changed:

  • Move to a buildout based infrastructure
  • Move to supervisor for starting up the web app.
  • Adding a service wrapper for elasticsearch, but elasticsearch is not controlled by supervisor,
  • Social media button!!!!
  • Moving code to use GPL for license
  • Change size of div to accommodate firefox view of pdf

The source code is at: https://github.com/sweemeng/Malaysian-Bill-Watcher

What next, bug fix, localization.

 

GADC KL 2012: The experience

So I have participated in the Global Android Developer Camp a.k.a GADC, the experience have been great. This years event in Kuala Lumpur is held in iTrain. Kuala Lumpur is one of many location that the event is held.

IMG_20120219_092025.jpg

GADC is an event where android developer comes together to create app or tools for Android. Other than giving us prizes, and talks on writing android app, this year we are given a chance to develop for the Smart Watch, only the winning team that develop on it can keep it though.

 

2-19-2012 12-33-41 PM.JPG
Team Plastique

A few team have created apps, 2 interesting project is Credit Card reward tracking, and a tower defense game. Team Plastique have created an reward tracking, and they manage to push it to the smartwatch, so they deserve the watch. Then we have a Tower Defence Game, which have a very interesting way to do in app credit. Ultimately the winner is Team Plastique.

 

 

Not surprisingly, GADC have more people trying to build their first mobile app, and a few that started out from iOS. Which left the organizer as the few people that have real experience in writing android app.

The event is a fun one, we can mingle around with like minded folks and share experience. The organizer are willing to share experience and knowledge. The only complain is, Sony Ericsson should do a better job setup the smartwatch as it would help us the test the code on a real hardware better.

12 - 1 Me, I didn’t win, but I did get a lot of knowledge and experience. I did finish my app. You can see it on the left, it uses JSch to connect to a linux box, and run a one liner, and diplay result to watch. It is buggy though.

 

 

Thanks to Nazrul, Wenjiun, Bernie, Mike for making this event great. The image I have on the post is courtesy of Nazrul and Wenjiun, I spent too much time hacking that day, sorry 🙁

Want to see more photo and related page go to

 

 

Android Interface With ADK: The Android Part

After hacking on the arduino site of the code, now we can go to the android part

For android side, it is a easier in a way. There is nothing to bypass, just make sure to have the right version of android OS. You need 2.3.4 above to be able to use the Accessory API. Though using the API have quite a lot of stuff to setup. And I am a bit confused with the documentation(which is why I share my notes here)

Screen Shot For ADK Test Project

Android Manifest

The manifest is in: https://github.com/sweemeng/ADK-Test-Project/blob/master/AndroidManifest.xml

It is pretty standard across all android app, except “com.android.future.usb.accessory”. Because I am still on Android 2.3.7.

Metadata

Then add a accessory.xml entry, under res/xml. in github https://github.com/sweemeng/ADK-Test-Project/blob/master/res/xml/accessory_filter.xml

Remember from the last post on arduino, the definition goes here.

Now to the code

The source code can be found at github: https://github.com/sweemeng/ADK-Test-Project/blob/master/src/adkTestProject/main/ADKTestProjectActivity.java

Library Used

Essentially openaccessory api open the device as a ParcelFileDescriptor object, which is why you need to import java FileInputStream and FileOutputStream, so you can read and write data to the arduino board

BroadcastReceiver is used because an Android Accessory can broadcast intent to android to launch an android app. IntentFilter and PendingIntent are used to setup  the permission necessary for an App is used. More on that later

Variable(For reference)

Now defining variable

Just so you can reference what object is the variable name is.

BroadcastReceiver

Remember an ADK device can send intent to an android app. So that when you plug your device, an app that is use to run with it can be launch. Essentially what this does is, when an intent is receive from the ADK, setup the app to read from ADK, via OpenAccessory method, or if it is detached, clean up all the connection via Close necessary.

line 8: Get the permission and action from the Arduino Board

line 11: Get accessory connection

line 13 – 14 : Get extra permission, from the Arduino Board

line 16: Open Accessory function to setup all the code and interface to the Arduino Board

line 27: from action UsbManager will check if there the arduino board id detached, then do cleanup

line 31-32: If cleanup is still exist, then call cleanup function

Initialize Accessory Interface

 

This is how you setup to read from ADK. Accessory will be defined from an intent from the ADK, mUsbManager will open the accessory, and return  ParcelFileDescriptor, which will be pass to FileInputStream and FileOutputStream.

FileInputStream is use to read from the ADK, for this project I use FileOutputStream to write to the ADK Board, which start the thread to have them run in background.

Close Connection to Device

Close is easy just close the file descriptor, and set variable to null

Send Commend to Device

Remember I say on the last post that,  you just send an array of byte or read an array of bytes , it is the same here.

line 4: define the holder for data to be send to the Arduino

line 7-15: just to setup the value send to be at most 255, the LED only can access such value. It is also nice to make sure value send from app to arduino, because once go to hardware, thing will be fried(shit just turn real!!!)

line 17-19: just to assign value, remember we send bytes to Arduino, so remember to convert accordingly

line 24: send the byte array to arduino

Retain Devices

Retain ADK accessory to be reuse by new activity

Initialize the app to use Accessory

This register the Permission needed and the BroadcastReceiver, also setup the retained Accessory object

 

To resume an activity

line 7-8: so the connection already closed, it is cool, don’t do anything

line 11-13: get all connected device, if there is nothing return null, else, return the first one(there is only one anyway)

line 17-21: permission checked out, you can open(OpenAccessory) and send value to the arduino(sendCommand)

line 25-29: request permission, again

Using the Using sendCommand

So just get value from UI change, then send the value to send command. It is a seekbar, that value is progress, and we already save color value anyway.

We’re done

I hope this is useful, though I think I might confuse people more, so just leave a comment if there is any question

Android Interface with ADK: The Arduino Part

IMG_20120212_143829.jpg

 

 

 

 

Update: Android site of the post is at here

I got myself an Seeeduino ADK board few months back, and I have rooted and flash my phone with cyanogenmod, just so I can get the right version of android OS, to use the accessory API. I have finally wrote some code, which I hope is useful as an example.

Prerequisite

There is a few thing that need to be done before one can start doing an Arduino side of the ADK project.

  • Fix the following line, in wiring.h. Detail below
  • use the older version of Arduino IDE, 0023. 1.0 don’t work yet with the ADK Library. New changes in the library breaks the code from google. So don’t use it.
  • I am using the Seeeduino ADK example. But I will highlight the part that is generic with all ADK project

Fixing wiring.h

Essentially what happen is, for a few linux distribution such as mint, ubuntu and debian, they have a upgrade to the avr gcc toolchain, one line in wiring.h is not compatible with it.

  1. Under your arduino folder, you will find file “hardware/arduino/cores/arduino/wiring.h
  2. Find the line with “#define round(x)     ((x)>=0?(long)((x)+0.5):(long)((x)-0.5))
  3. Comment it out, “//”
You are done. 

Let Start

The ADK is essentially just an arduino mega, so your arduino code just work. And the code is below. Or grab it from github, https://github.com/sweemeng/ADK-Test-Project-Arduino

There is essentially a few part that is essential in the Arduino side of an ADK project.

The necessary include for the ADK Firmware to work on arduino

The necessary definition. Please note that, ADKTestProject will be the name of my android app, and “sweemeng” is the name of the author that I use. I will cover this again when I show you the android app.

Serial is for debugging, not necessary(but it is nice for debugging), and the only line to initialized the ADK library is acc.powerOn(); .This will tell the Arduino board, it is ready in ADK mode. pinMode line is to initialize other hardware, in this case the RGB Led that I have

The data received will be in an array of byte, hence byte data[3], it will be the same as you send as well, but that is another story. Few line to note here acc.isConnected() to check if the arduino is connected to the android, acc.read will read into the byte array we define, and return a int which is the length of the output.

That is about what you need in the main logic inside the arduino.

  1. define an array of byte,
  2. check if it is connected with acc.isConnected
  3. if it is, call acc.read to read into array,
  4. check for error then execute thing that you want to do, in my case, blink an overly complicated LED
  5. Use serial to make debugging easy.

That strictly about it for the arduino ADK board to be used with android. Next up I am going to cover the Android side of the app.

ACTA like law in Malaysia

Updates: Added Letter from Malaysian NGO which include CAP of Penang, so not it is not just an issue for geeks

Again another set of law being considered, again it involved directly in our digital life. While the whole society still being fired up over the 1Care, and people still debate over the fight in a fast food chain.

Another story begin to emerge, the govt wanted to establish a US like IP law in Malaysia, because of the Trans-Pacific-Partnership agreement.

http://thestar.com.my/news/story.asp?file=/2012/2/9/nation/10692529&sec=nation

I am not going to explain more, because there is links that can explain this better than me

  1. The Electronics Frontier Foundation : https://www.eff.org/issues/tpp
  2. TPP Watch : http://tppwatch.org/
  3. TPP Digest : http://www.tppdigest.org/
  4. The document involved : http://infojustice.org/wp-content/uploads/2011/04/Koo-TPP-Section-by-Section-Analysis-April-2011.pdf
  5. Letter from Malaysian NGO : http://www.citizen.org/documents/Malaysia-TPP-NGO-letter.pdf

As a Malaysian, this will affect us. As an Software Engineer, now software I wrote potentially already covered by some overly broad software patent. As a linux user, the use of libdvdcss is potentially illegal. Among other thing.

p.s This post is not to undermine other issue happening in Malaysia, in Malaysia we have many problem, this is another one that potentially bite us HARD.

Python Malaysia Meetup Postmortem

A.K.A How to run a geek event in Malaysia

After the yesterday’s Python Malaysia meetup, here is a few thing I want to try up, or keep on using for future event

  • Malaysian tend to be late, so always put the time a bit earlier, at least 30 minute ahead, that is the usual time for Malaysian to be late
  • Half of the user do not attend, even though they have registered on eventbrite. What I want to try next time is, try charge money for the next event. The money will goes to pizza
  • The event is a bit bland, some suggest a full day just for Python User Group Meetup. I try to run it like most other python user group, one topic per meetup. Bigger event also harder to run. What I might test out, is having a lightning talk for each meetup.
  • We have our networking session in a mamak, we ends up using 2 rows of table. It is not a bad thing, though table kinda limit movement, maybe next time, we order pizza(Go to point 2). It is nicer for everyone have a chance to talk to each other
  • Python Malaysia need a proper website, not everyone using facebook. Even though most go from facebook event page to eventbrite page. Still it is a nice thing to have.
  • Location matters!!!!! ITrain is just the right place to have an event, in the middle of city, accessible via LRT. Car park can be a problem though. It helps to bring more people in, because of the location.

Btw, there will be another meetup, but that is next month.

Just another WordPress site