The Evolution of Pi - Part II

This is a continuation of Part I.

Another possibility came about arising from my son's science project:

Q: How do you test the capacity of [rechargeable] batteries?

There are multiple possibilities:

• After some research, I bought a battery capacity tester (DC 60V 100A RC Boat Heli Watt Meter Battery Power Analyzer Digital LCD Display) for $13.42. A quick testing found it wasn't accurate for AA batteries. For example, the current shown on the LCD display is way off when a multi-meter is wired in the circuit. 
• Then I got inspired by this Denis Hennessy article. There is no reason I can't do the same thing with the Pi with a few modifications
• I don't need the casing for the science project. I would also use a bread board instead of proto board since I fully intent to tear it apart once my son's project is over.
• Would like auto-shutoff at the drop off voltage. This will prevent over-draining the battery, which will kill the battery or cause other damage in the house. 
• Would also like some kind of notification.

The Design:

The diagram is functional yet not fancy.

• K1 is a manual switch -- the "master" shutoff.
• The Relay is controlled by the Raspberry Pi to close or open the circuit so we can programmatically start/stop the testing.
• The battery is drained through "R" (10.5 Ohm wire wound resister, mounted on an aluminum bracket as the heatsink)
• "r" and LED provide visual indicator whether the circuit is on/off. We also blink it anytime we sample the data.
• V-in0 is the input to an ADS1115 ($4.69 shipped from China). It is a four channel analog to digital converter that is a wonderful little device. The downside of Pi (compared to Arduino) is the lack of analog input, hence the need for something like the ADS1115.
• So the Pi eventually reads the voltage over "R", then calculates the current, wattage, and WaHr.

We also use a small fan to blow at "R" so it wouldn't over-heat, as well keeping the "10.5" Ohm accurate. Actually we adjusted it down to "9.1 Ohm" to count for the effect of "r" and the LED. The following is a picturing showing the resistor (R), the relay and ADS1115. The 8-channel relay can be replaced with a single channel one. But the price was right and we can use it for other projects later on. The white stuff between the resistor and the heatsink is thermal paste, conducting heat from the resistor to the heatsink.

The Coding:

We decided to use the Python RPi.GPIO library which works with the ADS1115. Once things are initialized, the code goes into a loop: measure the voltage, flash the LED, print out the values, sleep 60 seconds, and start over again.

When the cut-off voltage is reached, it turns off the relay, and prints out the final output.

Another nice feature is we used the Notify My Android application and the PyNMA client library to send notifications to Android devices. Works pretty well. The follow is one sample test:

Epilogue:
In my son's project, he would SSH to the Pi, run the command line, and walk away to do his things. I showed him to use GNU screen to prevent the program from terminating due to timeout.

At this point, Python turns out to be a very handy language for prototyping. You can start doing things with little planning, and learn while you experiment.

A few weeks after my son completed his science project, I dismantle the whole thing. The Pi continues to announce incoming calls as covered in Part I. 

Roll Our Own Internet Traffic Monitoring Platform

Q: How do we know a person (like our son) visits certain sites?

A: (Any good answer starts with a set of questions to clarify the intent)

• Why do we need to know? [just because we want to know]
• That is not convincing, any real reason? [like protection, such as malware, internet bullying, etc..]
• OK, fine, I guess there is a need.

Pondering on the possible solutions:

• Install anti-virus/anti-malware/anti-?? software on the devices. The problem is that the client device needs to support features like privileged accounts or parental controls. It effects performance (we dislike the anti-virus software enough on the desktops). It only works on the given device.
• K9 Web Production came closest, given its excellent light weight nature. But it didn't have a good mobile version for tablets, also lacking was family support of multiple devices. It also work in the "prevention/blocking" way, instead of passive monitoring.
• A more robust way would be a network based solution.
• The home routers could have a package to provide individual traffic monitoring with some analytic s capabilities. It turned out not so (as of 2014).

The decision was to roll-our-own solution. Besides it being cool, it is also a great opportunity to toy with different technologies (DNS logging, remote log, python, PHP, PowerView, etc). Building blocks we have at hand and constraints

• Firewall router running OpenWrt. 
• A Synology DS101 disk station.
• Nothing else: Decided NOT to build additional *inux servers.
• Shall we say -- low cost?

The solution can be shown in two views.

Functions View: The following view captures the intended functions of this solution, INDEPENDENT of special tools and/or techniques. This is one of the standard ways we try in the "Solution Architecture" practice, to show "what" aspect of the solution.

Component Interaction: The following view shows how this solutions would work.

• Steps 11,12,13,14 are the standard user experience: A user with a client device accesses the internet, including the DNS lookup, and the general firewall protection. The only change is the enabling of "Local Logging" of DNS (step 21), with no perceivable user impact.
• The "DNS Query Forwarder" Python script starts as the firewall powers up (step 22), reads the kernel log using the "logread" utility (step 23), identifies DNS query entries, and forwards to the "Synology Disk Station" syslog facility using netcat (step 24).
• On the Synology Disk Station, the log entries are rotated into daily files. Log rotation makes the next step more efficient.
• A daily cron job is scheduled to run right before midnight, to read the daily log entries, and looking for the DNS queries (step 25).
• In step 26, the domain names are trimmed for efficiency. For example, us.www.yahoo.com, and images.www.yahoo.com will both be trimmed to www.yahoo.com as both subdomains will be of the same category.
• In step 27, the domain "type or category" is determined. It first looks at the local SqlLite database for a cached copy. If the cache expired or doesn't exist, it connects to WebRoot to determine the type, and cache the result. This is a little more complicated as the "list of categories" need to be retrieved during setup, and refreshed later as necessary.
• In step 28, the enriched query records (now with domain type) are consolidated into a daily file. Other information includes client host name, timestamp, etc.

The parents (31) later can use Excel with the PowerPivot extension (32) to pull down the file, and do analysis. Of course a template is created to make this a repeatable chore.

A Few Limitations:

• This solution only identifies the devices, not exactly the person. Tracking at a person level would require a user identity
• The "amount of traffic" is also not accurate. A client device may do a DNS query once for a website, and use the result during the TTL time (time-to-live) for any number of visits. The number of visits are not captured in this solution.

Afterthoughts:

• We ended up looking at the dashboard much less than we could. The dashboard is interesting, but not really actionable. Amount the mountain of data, the actual high risk items are very small, and could hardly tell anything in isolation. For example, a "gambling" website may just be advertisement.
• Python is a great language for prototyping as it is an interpreted language. But I really don't like the use of indentation for grouping statements. In comparison, I found PHP is closer to modern programming styles (like Java or C++). 
• After 6 months, we disabled it so we don't pay the API fees. Now we know, we can activate it anytime we want, if ever so.

Architecture Musings: This paragraph is more tolerable since we even bothered to put the design in a nice format. The question that must be asked, is, could there be an easier/better way?

• Steps 1x, 21, 22, 23, 24 are pretty lean. Using a python script to read the log and forward over is as easy as it can be made.
• Instead of using Syslog, we "could" have used Apache Plume, and sink the output to a file. But that would be equivalent to Syslog anyway.
• Of course, we could be more ambitious by using Flume to sink the data into Hadoop (instead of syslog), and Oozie as the scheduler (instead of cron) to create datamarts, which can be analyzed/viewed later. If so, we can even write a custom Flume interceptor to figure out the domain category on the fly. This is a serious over engineering.
• Another possibility is to send the log files to AWS S3, and use something like lambda to trigger data enrichment and analysis , which could be dashboard enabled. That would be yet another day.

The Evolution of Pi - Part I

A few years ago, I bought a Raspberry Pi model B for my son as a Christmas gift, hopefully he would show some excitement. Understandably now that it didn't turn out so.

It was in the closet for a few years, then a need for a project surfaced:

We often get phone calls during diner time. We either rush over to the phone and found it was a telemarketing call, or sitting and worrying that we might miss an important call. Wouldn't it be great if the phone can make voice announcement?

There are multiple possibilities:

• We could buy a phone with such features, but that would be lame, Plus it would be against my philosophy of doing more with what we have.
• Very few phones could announce both the number and name from the caller ID.
• It really costs money if you want to buy a cordless system, with voice mail, etc.

Why not use the Pi (from the closet) to monitor the phone with a modem, and send the voice output to a speaker?

• The only additional thing we would need was a USB modem. The Trendnet TFM-561U looked like a perfect little device. The price was great.

The Solution Details:

The Pi needs to be connected to some audio device, that is loud enough so we can hear easily at the other end of the house. The Pi would quietly monitor the phone line via the modem, When a call comes in, it would announce the caller ID (number & name) to the audio output.

We have a pair of cheap PC speakers, and we could switch it on/off with the Pi using a relay. But I must concede the voice has a lot to be desired :-(  Then, why not connect the Pi to the Yamaha RX-V473 receiver -- great sound and can be as loud as we want. There are a few lingering challenges.

1. For this to work, the receiver needs to be on. But we don't want to leave the receiver on all the time just for the few phone calls.
2. Also, if the receiver is set for a different source (like watching a movie) and a call comes in, it would ignore the announcement.

Some research (and reading the manuals of the RX-V473) discovered that it is a rather intelligent device. Not only it is network enabled, it also has API so you can control it. Even better, there is a python library called rxv that is readily available.  We also added a few nice features

• Save the current state (mode/source, volume, etc) and restore it at the end of the caller ID announcement. 
• Use Google Translate API (translate shell) to transform a text string to voice. We also decided to enjoy English, Chinese, and Spanish for most part.

Here is the code for the script:

#!/usr/bin/python
#usage: yamaha-rx-v473.py [translate string]
#ex:    yamaha-rx-v473.py  #announce in English
#ex:    yamaha-rx-v473.py  EN=zh-CN #announce in Chinese

import rxv, shlex, subprocess, sys, os, time
receivers = rxv.find()
rx = receivers[0]
# Save current stats
save_power = rx.on
save_volume = rx.volume
save_input = rx.input

if not save_power:
  rx.on = True

rx.input = 'AUDIO'
rx.volume = -22

#read standard in into an array so we can pipe it out multiple times
list=[]
for line in sys.stdin:
    list.append(line)

#play the message, trans automatically handle multiple lines of inputs.
FNULL = open(os.devnull, 'w')
subargs = shlex.split('/usr/bin/trans -p -player /usr/bin/mpg123 -b')
subp = subprocess.Popen(subargs, stdin=subprocess.PIPE, stdout=FNULL, stderr=FNULL)
for line in list :
    subp.stdin.write(line)
subp.stdin.close()
subp.wait()

if len(sys.argv) == 2:
    subargs = shlex.split('/usr/bin/trans -p -player /usr/bin/mpg123 -b ' + sys.argv[1])
    subp = subprocess.Popen(subargs, stdin=subprocess.PIPE, stdout=FNULL, stderr=FNULL)
    for line in  list :
        subp.stdin.write(line)
    subp.stdin.close()
    subp.wait()

#sleep for 3 seconds between swiching back. So the receiver isn't turned off 
#too early before the announcement is over.
time.sleep(3)

#Restore the saved values
rx.input = save_input
rx.volume = save_volume

#Send system standin to subprocess and play it
if not save_power:
  rx.on = False

Tying'em Together:

Of course, there is also the code of interacting with the modem. For that the NCID package works perfectly as advertised. In particular, we configured the ncid-speak to invoke the custom python script. Of course, we created the rc scripts, and "update-rc.d" to start ncidd and ncid-speak at boot time.

Reflections:
We have had the system up and running for about a year now (as of Feb 2016), by and large it worked great. We had to update the translate-shell a few times, as Google changes its API. As good as it is, it takes the on-going feeding, which is beyond the appetite of most typical households. So buying a phone with voice announcement might not be a bad idea. Pulling all the pieces together was great.

Evolution of a Timer Switch

The 2/9/2016 Problem: Following are the later occurances

• 05/04/2018
• 04/30/2018
• 04/27/2018
• 04/19/2018
• 04/13/2018
• 05/19/2017
• ?/?/2017
• ?/?/2017
• 12/19/2016

11/22/2016 Update:
The same problem as 2/9/2016 happened today.  Since the original posting (on 2/9/2016), I have encountered the problem 4 times in total, while using it once daily. This translates to a failure rate of 1.39%.

• The most annoying part is knowing how to reset the timer, when it happens. Pushing the ON trigger switch followed by the OFF trigger switch does the trick.
• Also, the severity of failure. In my case, more coffee than I need for the day. 
• Though I bought a replacement unit, I decided to delay replacing the current unit.
• The problem with the current unit it does not happen often, and there is little damage when it does happen.
• The new unit may or may not work better.

7/22/2016 Update:
The same problem as 2/9/2016 happened today.

2/29/2016 Update:
The same problem as 2/9/2016 happened again. It it is more often than once in a blue moon (actually twice over the past month, though not blue moon).

Some analysis is needed to pinpoint the failure of the solution. It could be the timer, or the red start switch being sticky (refer to the timing chart). If the red start switch is sticky, wouldn't the reset button stop it anyway? To test that theory out, I pushed and held the red switch, and then pressed the reset switch -- it buzzed (may be angrily). Since I didn't remember the buzzing sound, so the start switch was fine. But I could have missed the sound with the grinder going, or due to the attempts to stop it. By the way, I used a lamp for the test, and knew that the reset would NOT work if the start switch is pushed down. So the test is inconclusive.

At this point, I couldn't think of a way to rule out either the time or the switch. Given that fact that the switch is much simpler component, the chance of the timer going bad is much higher.

It is conceivable I can buy a different (and hopefully better quality) timer. But decided to give another chance to the ST3PF. We all deserve a 2nd chance, don't we?

I must concede this troubleshooting is a bit shoddy.

2/21/2016 Update:
I ran into this article that explains the different types of timers really well. I wish I had the information when I started the project.


2/9/2016 Update:
This morning, the timer lasted way longer than 10 seconds. Even hitting the reset button didn't stop it, so I ended up with more coffee than needed. Hopefully it was a one time fluke, which is hard to imagine. We will wait and see whether it occurs again in the future.

Original Post
After multiple failed attempts to grind coffee consistently, I finally decided to splurge and bought the Baratza Virtuoso, which can grind the exact amount of coffee each morning at the specific coarseness.

It takes one (1) day to realize the dial switch on the grinder is completely useless. It is simply not practical for easy and consistent setting. For example, I need 10 seconds (the magic number deserves its own blog), and there is no marking of such. Even if I can find exactly where it should be, and note it with a permanent marker, doing it each morning half awake before coffee is humanly impossible.

Well, I can make a timer -- actually a timer relay with delayed off,  called "Off-Delay Timer" to be specific. Hence the Fuji ST3PF came to the rescue. It has an adjustable delay of 1-60 seconds, with a start and reset trigger. I bought it on Oct 28, 2014 at ebay for $7.16, including shipping and handling, from China. How could they make money is still a mystery. I could have bought more, as it is selling for about $12 in Feb 2016.

I decided to retro fit it with a power strip, and you can see the finished product.

• The timer is in the middle of the power strip.
• The start (red) and reset/stop (black) buttons are at the right end.
• The outlet on the right one (1) is controlled by the timer relay - no need for 2 or more.
• The left 2 outlets (one plugged in with the blender shown at the front) are not controlled by the timer.

It was a resounding success. Without saying, there was some necessary calibration (how much coffee ground is produced at different coarse levels settings on the grinder in a given time like 10 seconds, which can be used to determine how much time is needed for 28 grams, since we decide to be exact).

A few months later, I found out the timing wasn't correct, when I started using a moka pot, which needs less coffee. After some investigation, it was discovered that the marks on the dial is NOT what I thought. For example, setting it to "2" does not mean 20 seconds.

That needs to be rectified. After about an hour of re-calibration, you can see the marks and numbers I wrote with a marker. Notice how far away it is between 20 and 30 seconds? Amazing.

Anyway, now we have a piece of improved high accuracy equipment, honoring the daily coffee ritual (manual drip on weekdays, french press on weekends,and moka pot on random days).

Reflections:

• Even with the manual re-calibration, I would buy the timer again in a heart beat. It is a lot of time for the money (pun intended).
• It took some research to find out and decide exactly what I would need. Initially I was thinking of Arduino plus a relay, or maybe even a Raspberry Pi plus a relay, or a- dial timer switch as you see often in the bathrooms of campsites. It ranges from micro-controller based DIY projects, to electrical gadgets. The final narrowing down to the category of "Off-Delay Timer" was a significant step.
• Then choosing the right type also took research. Different vendors have very different products and prices. Even the Fuji Electric catalog shows a long list of products of various specifications and prizes. Reading through the literature, and understanding each, was essential to picking the right one. For example, you need to look at the timing charts carefully to decide which specific one you need. Then you can decide whether you want the 5 second, 30 second, or 60 second models. No, "Chinese" is not the problem, it is the reality of discovering and navigating the complexity when transitioning from a concept to a real thing. [revision: Knowing Chinese helps. The Japanese version is much harder to understand]
• Critic of the ST3PF: As great and cost-effective a product it is, I wouldn't call it high quality. The marking not being accurate is a HUGE minus, especially since it is not noted (therefore leaving customers to discover). It is rated at 20 million cycles, which translates to about 55 thousand years. It will be fine even if it can last a fraction of it.
• On the other hand, the final product I created didn't have any specifications, little testing, and no manual at all. 
• An "industrial product for sale" needs a lot of rigor and work, beyond "something that works," even with all the limitations.

On Coasting Downhill

Q: Do you coast downhill faster/farther if you are heavier?

After an engaging discussion of the whole family..

The short answer is, a heavier person will coast down hill faster than a lighter person, if all other factors are the same.

A person coasting downhill has 3 forces working on him and the bike.

• F1: A downward force parallel to the surface. F1 is proportional to the weight. (F1 = cos(angle) * M * g). Given the same hill cos(angle) is the same, and "g" is a constant, the heavier (M -- mass), the bigger F1.
• F2: The rolling resistance. I can't speak of it thoroughly, so I quote wikipedia. Basically, F2 is also proportional to weight.
• F3: Is the air resistance, which includes air displacement (like bumping into somebody) and friction. Both of these factors are related to shape and speed, but has nothing to do with weight.

So the acceleration force F = F1 - F2 - F3 = M * a.

• If F3 = 0, the acceleration will be the same for everybody, since F1 and F2 are proportionate to M.
• When F3 > 0, its net negative effect to a lighter person is greater to a heavier person.

Solely and only due to the effect of F3, a heavier person is impacted less by air resistance (comparatively speaking), and therefore coasts faster.

A side note on F3, specifically the disproportionate impact to a lighter object. The air displacement is the amount of air an object pushes to the side. This is the impact if I decide to rush through a crowd of people by knocking everybody down, of course with a few bruises on myself as well. However, my bruises are smaller if I am running through children (now all moms hate me), compared to running against a crowd of football players. By the same token, a bigger car, SUV, 18 wheeler, etc, has more advantage in an accident. All in all, slowing down more when coasting down the hill is very mild compared to other unpleasant consequences.

The above statement is corroborated by the advice given at Pinewood Derby races of cub scouts -- you shall make your car as heavy as possible (within the limits), and put weight as farther back as possible without losing balance.

• Since the car coasts downhill, the heavier the car is, the faster it goes.
• Arranging the weight to the back achieves the effect of raising the center of mass higher, since the race starts at the higher end of the track. This arrangement makes absolutely no difference all the way till the front of the car reaches the bottom of the track.While the car slides from a downward incline position to a level position, the weight at the rear provides additional potential energy to continue accelerating the car, compared to the ones that front-load the mass. This is the reason you see some cars "jumping ahead" of others the moment they reach the bottom of the ramp. Just so now you know.