Making your first Capsule for Samsung Bixby – An Exercise in Teaching your Phone to Listen

 

Today Samsung made the Bixby Developer Studio available for download and use so that developers can start building capsules to publish on their marketplace starting in 2019. I am an early adopter of the new Bixby and wanted to share how to build a simple capsule using the new developer kit as well as share my experience with using the new platform. Readers of this blog know that I have made and published Skills for Amazon’s Alexa and published tutorials on how you can develop for that platform. Similarly this writeup will focus on a minimal application that will help you get started with the features of Bixby’s impressive development tools.

Bixby is a wonderful platform to develop for and it has top-notch development tools. Building software for Bixby is a lot like teaching someone a new skill. It uses natural language model training so you can show Bixby what parts of user phrases are important and it uses the idea of concepts to define Bixby’s understanding of what capability you are giving it. These concepts will be discussed in detail below.

Today we will be making a capsule to generate passwords made up of a random string of words, inspired by XKCD’s Password Strength comic. We will be taking advantage of Bixby’s visual interface to make a password users can easily remember as well as easily copy and use for their accounts. The XKCD algorithm sticks random, memorable words together so that passwords are complex but can also be easily recalled by the user. They also are higher entropy than a random string of characters and numbers, making them harder to crack my many brute-force methods. After giving the comic a quick look, read on.

As always, you can download and view the code on my GitHub!

THE PROBLEM

We want to build a Bixby capsule that can generate memorable passwords for users. These passwords should be of a user-specified length.

The overall requirements are:

  • Generates a password using regular English words
  • Takes in a user’s specified length
  • Displays the password graphically for copying
  • Displays a calculation of the entropy of the password so the user knows how good the password is.

THE SOLUTION

You should now have Bixby Studio installed on your system. As of writing it is available for Windows and macOS.

Create a new project by clicking File>New Capsule.

The first bit of code we will focus on is the generator.js file. This is where we define our entry point and what we are going to return.

Notice how we export the function generate- this is the function where I generate everything we need for the response you see in the screenshot above. We take our wordlist dictionary file (how to get that will be discussed shortly), we build our password using a user-specified length called numWords, and we calculate the entropy of the password. We then return a result we can parse into a nice, visual response like in the screenshot.

Whew, there’s a lot going on in here though! Let’s start with the wordlist. This is a JSON-formatted list of common English words I found searching for open-source corpora. Why JSON? As you might have surmised from the above code snippet, Bixby capsules are written in JavaScript! Importing this data as JSON makes it very easy to loop through and use, as you can see in my generate function. I stored this in a directory called lib but you can call it whatever you please. Just be sure to update the path in the generator!

Next, we need to discuss how we get numWords. This is the user-input. We want the user to say ‘Make me a password with three words’ and Bixby needs to know how to do that.

In the resources directory you will find endpoints.bxb. The actions your capsule can take are called endpoints. Let’s define one for generating a password:

Let’s look at what we have here: We have authorization set to none because this endpoint is public and available to any user without authorization. We have specified an action endpoint for our generate function as defined in the generator.js snippet above and we have told Bixby that the input for this endpoint is numWords. We also tell it what file it will find the definition for this endpoint in- generator.js.

Now that Bixby has an available action in the form of the endpoint, we get to the really interesting stuff- teaching Bixby what everything in our capsule means. The way we do this is via a model. In the model directory we have actions and concepts. These make up Bixby’s understanding of what your capsule can do, and we just need to write some high-level markup to make this work. Let’s start with the action our capsule is going to have- generating passwords. This will inform what concepts Bixby needs to have definitions for so that we can move on to training our natural language model.

Above is the generator.model.bxb action file. You will find it in my action directory. What does this do? Read through the comments carefully. It defines the actions Bixby will take when running this capsule, and it covers all our bases regarding various user inputs! We tell it our action is to run our generate function. We tell it to collect numWords, and we tell it that numWords is of the numWords concept type which we will define shortly. We tell Bixby that there can be at most one numWords (so that we ignore other numbers in the user’s invocation) and we tell Bixby that this value is required. If Bixby cannot find a number in the invocation to use, we define a default initialization with four words- the same as our XKCD comic! We then do some validation in the event we find a number in the user’s invocation. If numWords is 0 or less, we want to display some text telling the user that you cannot have a password that is negative in length (duh, but the bulk of software development is anticipating stupid). Finally we tell Bixby what our result is going to be- an instance of our PasswordResult concept, which will be of the type Calculation. This is a type Bixby provides for a result that it needs to compute or otherwise derive. Let’s get started defining what these concepts are.

If you are following along in the repo, look at the numWords concept.

This is a good minimal example of a Bixby concept. These are the variables that are key to our capsule working. You can think of them as teaching Bixby a new idea, slowly building for it the picture of what you are trying to achieve. We tell Bixby that NumWords is an integer (we don’t want fractional words). We also give a brief description of what this has to do with our capsule. For NumWords this is obvious- it is the number of words in the password.

Password is almost the same except this concept is given the ‘name’ type since we need an output string. We describe it as the output password. Entropy is similar- we describe it as the approximate bits of ‘randomness’ in our password and give it the integer type since it will be a number we calculate. Length, predictably, is an integer that represents the length in words for our password. This is utilized in the entropy calculation, which taking the formula from the comic is taking two to the power of the number of words and then dividing for the number of attempts to brute force the password you could make if your computer could make 1000 attempts per second for a year. This yields an estimation of the number of years the password would take to crack in these circumstances. Finally Years is given the integer type and described as the number of years simple brute forcing would take to crack this password- it is also part of the entropy calculation we display at the bottom of our result as you can see in the above screenshot.

The most complicate concept is our PasswordResult:

It has the type Structure because it contains multiple properties- namely every concept we have just defined. We give these properties types- I just made these the same as the property name for simplicity but they can be used in more complicated capsules to link properties together with a descriptive type. We again describe each property and what it does, tell Bixby if the property is required, and for each tell Bixby that there can be at most one value for each. This result, as you may recall from the generate method, is what we will use to generate our visual response on the screen of the device. We have now explicitly told Bixby everything there is to know about how our capsule is going to work! It knows every concept and every result we are going to want. We now can teach Bixby how to handle speech.

Click training in the resources/en directory.

Screen Shot 2018-11-07 at 7.11.44 PM

You will see a list of training examples I have provided the natural language model. We are effectively training Bixby to understand how to parse user phrases and turn them into useful input for our capsule. This is an application of machine learning! Notice the examples I have provided. I have made one: ‘generate a password for me’ with no numbers in it- this is to provide an example where Bixby should use our default input of four words from above, like the XKCD comic. I also provide numerous examples with varying numbers of words asking Bixby to generate a password in various ways. Notice how I have clicked on and highlighted the number in each training phrase and I have labeled this value as numWords! You will do this for each input your capsule needs- the more examples the better. Bixby will use the labels and examples you provide to teach itself that when something sounds similar to your examples Bixby is being asked to open your capsule and feed the data that is similar to the labeled phrases you gave it to the capsule as input. Bixby is learning, so make sure to spend plenty of time here to make sure Bixby really gets it! Compiling the model will make Bixby learn each of your examples and you can view what Bixby’s output for your examples would be so you can be sure that Bixby has not mis-learned how to handle your examples. A well-trained model will make your users happier and your capsule easier to use. This is my favorite part of the Bixby developer tools- it is very intuitive and fun to use, and it offers a look for machine learning enthusiasts into the underlying technologies behind Bixby. This is a defining attribute of the platform for me- it feels much more flexible than Alexa, which as a developer seems to encourage a more robotic and specific interface for its skills than the more flexible Bixby interfaces for capsules.

With your model trained and your concepts laid out, the last thing to do is to specify how Bixby should display our output. This is done with dialogs and layouts.

Dialogs define for Bixby’s interface the concepts (inputs) and the results. Therefore for each input you need there will be a dialog and for each result there will be a dialog.

NumWords therefore gets a dialog like so:

This is pretty bare-bones: We define a concept dialog (input dialog), tell it to look for NumWords (like in our training!) and we provide some template text for this type if we wanted to display something related to this input (in my project I ended up not using it).

The Password Result Dialog defines the dialog for our result. This one is more important for this project as it will populate our layout.

We define an output (result) dialog, have it match this time for our PasswordResult concept (passing in the output from calling generate with our numWords result) and then we tell Bixby what to write on the screen with the template text: Notice that this is the first bit of text in the above screenshot that appears when Bixby is displaying a result telling the user what it did for them!

The layouts for the visual part of the display (like this one, PasswordResult.layout.bml) look a lot like HTML! There are many documented UI widgets you can use such as pictures, hyperlinks, cards, and more. Here you can see we use a card to display the actual password, making it wrap onto the next line for long passwords and making them easy to copy. Down below in a div tag we display the password entropy. This is calculated using the formula from the XKCD comic, as described above. Finally we hyperlink to the comic that inspired this project as a way of giving credit.

A few more example passwords are shown below:

You can try it out for yourself in Bixby Studio! Simply click the icon that looks like a phone on the left hand side of the screen to open the Simulator, giving you an idea of what your capsule will look like on an actual Samsung device when the marketplace opens in a few months.

SHARING THE SOLUTION

This project can be found in its entirety on my GitHub! I hope this very early tutorial can help developers make their first steps into developing for Bixby, which I think has some very compelling development tools and technology behind it.

How to download a YouTube video for Seesaw

Have you ever wanted to download a YouTube video to share elsewhere but found that every ‘YouTube Downloader’ on the internet is an adware-filled, slow-to-load piece of junk? This is especially problematic when you are trying to get video content to your students- most of the time you cannot directly link to YouTube on a school network connection as it will be blocked, leaving your only option to download the video and re-share it on a platform like Seesaw, a popular platform for student engagement. So how can you get the raw video file to share without getting your computer infected? I have designed a simple desktop application that downloads YouTube videos and I am offering it for free to download! It is available for MacOS and Windows. Best of all it’s a portable app- meaning no installer and therefore no need for an administrator password to use on your school computer!

If you are interested in how I developed this program, read on.

THE PROBLEM

YouTube download websites are filled with adware, malware, deceiving links, and often inappropriate ads. Educators need to be able to download YouTube videos to get around school content filters. They need to be able to use a program that can also evade roadblocks like not being able to install software on a school-issued computer.

THE SOLUTION

screenshot
The Windows version of the application

The solution was to develop a desktop application (ensuring the longevity of the project as there would be no need to pay for hosting a website or risking my web application getting blocked by a content filter). I also wanted to write this application once and deploy to my two target platforms – Mac and Windows. There are various libraries for downloading YouTube videos simply – I figure a lot of the scammy websites are making use of the open-source ytdl-core or similar JavaScript libraries to download their videos and quickly deploy new websites. I wanted to use something similar as these libraries are easy to use, well-supported, and cross-platform. The solution for all of my problems was to build my app using Electron, which lets you build cross-platform apps using HTML, CSS, and JavaScript. I could build a website using ytdl-core to download videos using the same technology as the sites people are being driven to by Google and then make it a desktop application. Then, using the exact same code for both platforms, I could then deploy to Windows and Mac! And that’s exactly what I did- what you see above in these screenshots is literally a simple website using jQuery, Bootstrap, and ytdl-core that takes in the YouTube link the user pastes in the box, uses regex to check that the link is a properly formatted YouTube link, and then opens a save prompt to save the video to the user’s computer! The save prompt is one of the neat elements of the Electron API that you can just use and have it work one each platform, pulling up a native save dialog and letting me configure it to ensure the user can only save MP4 files, for example, as that is the file format Seesaw accepts. The open-source Electron Packager then lets me bundle all of my dependencies together and create a Windows and Mac application bundle. These bundles include everything needed to run the application in exactly the spot the application needs it, meaning there is no installer to run and therefore it can run without administrator access on your computer! The whole ecosystem let me hack this together very quickly with almost no friction between any of the tools.

ytdl-core lets me ensure that we automatically grab the highest-quality version of the video and download it in MP4 format, making sure everything is as simple as possible to go from video to Seesaw fast, saving teachers time. This all happens in the background – the goal was to make a tool that takes minimal thought to use. Just grab the video your students need, and paste the link!

SHARING THE SOLUTION

This application is completely open-source. You can view it on GitHub here! This also serves a project homepage you can share with your friends and colleagues as it features download buttons for both Mac and Windows. It is also where you should go to get the most up-to-date versions of the software- as I improve it and make bug fixes the links below will not be updated, but they will always get you the first release.

V1.0 for Windows can be downloaded here directly.

V1.0 for MacOS can be downloaded here directly.

 

A scrolling LED Display for your Google Calendar!

My brother had the awesome idea to stay organized this school year with a scrolling LED display that would read the next five tasks off a Google Calendar. This is the exciting end result! The LED display we used unfortunately ran only proprietary software called PowerLed, giving me no way to programmatically set the text the screen was displaying and instead requiring each line be manually entered (like some kind of neanderthal). Fortunately Python has tools for automating actions in even the gnarliest GUI programs, so I wrote up this guide about how I managed to make this dumb device smart!

THE PROBLEM

It’s <current year> and manufacturers are still shipping hardware with awful proprietary software and no way for the end user to tinker with it. Additionally it is not web-friendly and all information has to be edited by hand.

The specific sign I worked with was this one although many signs originating in Asia ship with the same PowerLed software, so this guide should help you out! I made use of the WiFi interface, so if your sign does not have one you should check if it has a serial interface instead and make a few modifications to my program (remove the Wifi stuff, have the program click the button to send the configuration over USB for example).

THE SOLUTION

(TL;DR here is the code!)

There are a number of tools that let you click through GUI programs and automate tasks with Python, but today I am going to focus on pywinauto, a popular set of modules for automating tasks in Windows GUI programs. This is appropriate because as far as I can tell my sign only came with a Windows version of the software (although there are mobile apps that work with it supposedly). Pywinauto works by opening a program at a specified path and then navigating down menus it can see. For example, I can tell it to send my configuration by telling it to click a certain member of the Tools menu at the top of the screen: ‘window.menu_select(“Tools(T)->Send All”)’. It’s that easy! Those names were copied directly from the GUI itself.

I made use of the Google Calendar API which lets me programmatically pull down information from a Google Calendar. At the time of writing Google gives a generous one million queries per day which I cannot imagine one person would ever be able to exceed. To be safe though, my code updates the sign’s text every three minutes (although this can be changed at the top of main.py). The program I wrote pulls down the next five events on your calendar and processes them to be displayed by the sign.

How does this all work?

  • PowerLed uses ledprj files which are essentially just XML files. I made my own template with placeholder text (‘REPLACE1’, ‘REPLACE2’, etc.) and saved this template alongside my Python script. This template let me play with scroll speed, font size, and other settings using the PowerLed software and trial and error with my display. To automate updating the template with Python, I simply read in this template file and do a find/replace for ‘REPLACE1’ and so forth with the five events I pulled from Google Calendar. When PowerLed reads this file, these strings are sent instead!
  • The LED sign broadcasts its own WiFI network that I must connect to in order to update the device. I use subprocess calls to netsh to drop off my WiFi and then connect to the display.
  • I use pywinauto to open PowerLed, which I have already pointed at my LEDprj file, and connect to the device. I then automated the process of pressing the ‘send’ button to update the display. Once updated, pywinauto automatically closes PowerLed.
  • The project then waits three minutes and pulls down the next five events again, updates the sign, and so on until the program is stopped.

By noticing that the LEDPrj files were really just XML files I saved a great deal of time reverse engineering how I can get Python to update this sign. While there is no API or command line interface for the software, pywinauto let me quickly get the computer updating the sign itself.

Now I am sure you’re thinking ‘This is all fine and good, but do I need to let this program run forever, popping open this PowerLed thing every three minutes?’ – the answer is yes. But there is a solution! Upcycle an old laptop (68% of Americans have an unwanted computer in their home) or make use of a cheap Intel Compute Stick or similar device (make sure it runs Windows, the cheapest ones run Linux). I used an inexpensive Windows compute stick with an Intel Atom processor inside that more than handily can run this program on loop forever.

How do you get this set up for yourself? Head on over to the Git repo and check out my README, it’ll fill you in! The gist is to clone the repository, follow Google’s procedure to enable the Calendar API and get a credentials file, change the values in the program so that it knows what time zone you are in and what networks it needs to connect to (I don’t pass credentials, the program expects you to have connected to your sign and your home network before) and then give it a try!

TROUBLESHOOTING

  • I assume you have PowerLed installed and you made sure the path in the program is pointed at where you installed it
  • I assume you have connected to your sign and your home network before
  • I assume that you have loaded display.ledprj in PowerLed and then exited the program so that it will load next time you open PowerLed
  • I assume you are running Windows and that you have installed Python 3

SHARING THE SOLUTION

My code and instructions can be found here.

Here are the exact products I used:

How to extract PDF file attachments using Python and PyPDF2

Tl;dr: Cut and paste the function I wrote here.

This is a quick technical writeup to hopefully answer a question I’ve seen posted a few times around StackOverflow and the issue trackers of various Python PDF libraries. This is especially handy for those of you who don’t want to dive through the PDF32000 to figure out how Adobe wants us to handle attachments.

PyPDF2 makes working with PDFs easy, but you may have noticed that it only has an addAttachment() function, similar to many other PDF libraries I tried. How do we extract attachments so that we can work with them? Embedding files in PDFs is very common and it would be nice to be able to interact with these objects, like we can with form fields and other things you might find in PDF files.

Fortunately the building blocks how how to do this are already available in the PdfFileReader class!  We just need to stitch them together:

  1. Read the PDF file using PdfFileReader from PyPDF2
  2. Decrypt the PDF if necessary (required, you can’t get to the embedded files without doing this)
  3. Retrieve the file catalog by retrieving the file trailer (reader.trailer[‘/root’])
  4. Navigate in the dictionary this returns to ‘/EmbeddedFiles’
  5. Loop through the list of files that are found there
  6. When we get to an IndirectObject, we have our file parameters. We call getObject() to return the parameters dictionary, then navigate to ‘/F’ where our file data is stored as yet another IndirectObject. Here we simply call getData() and get a byte string back. This can then be written to a destination file or processed however you please!

As always it’s better to show the code, so here’s a proof of concept script:

Easy, just not immediately intuitive when you want to do this fast! I created pull request to hopefully get this function added as a method for the PdfFileReader class.

 

The awesome soda crate shelving cart I built for my dad this Father’s Day

My father loves to collect vintage wooden soda crates. These used to be used to deliver soda bottles to grocery stores, and many were lost either due to rot or because they were replaced with plastic crates and tossed. They feature beautiful vintage logos and artwork and serve as advertising pieces. Today they are highly collectable, and make for some fantastic up-cycled storage.

I wanted to build my dad a set of shelves that he could use to display his crates as well as use them functionally as storage. As usual, I jumped to my favorite computer aided design program, OnShape, to start sketching up the perfect soda crate shelving cart. First however I needed to know what I was going to make the shelves out of.

My dad had found another design online where the builder had welded a cart together that he really liked, and it had an awesome industrial look. I am not a welder, but I loved working in the machine shop in college. One of my favorite things to build with is 80/20, which is pretty much a heavy-duty erector set for adults. We used it to hold up heavy vacuum chambers and other equipment, but 80-20 also has a lightweight series called Quick Frame which features 1″ by 1″ aluminum extrusions and is far cheaper than its industrial counterpart. These square extrusions can be stuck together using a variety of plastic connectors, and there are extrusions with flanges perfect for holding up the crates. Even better they offer cheap machining services which means that I could literally design a complete shelf on my computer and have them cut each piece to size and ship it to me – I would just need to assemble it!

Also working in my favor was that the crates are roughly standardized- roughly each one is 4″ tall, 18.5″ deep, and 12″ wide. This let me assemble a repeating pattern of aluminum bars with flanges on the sides to support the crates, and design for having six inches for each shelf, giving 2″ of additional storage above the crate. I also added caster wheels to the bottom so that the shelves could be mobile, adding to the industrial look. To finish the shelves off, I added a wooden shelf on top to store the odd-sized crates in my dad’s collection. The final design ended up looking like this:

Screen Shot 2018-06-12 at 9.16.37 PM
A roughly dimensioned drawing of the shelves.

OnShape let me render this design using RealityServer, which gave this gorgeous render of what the eventual shelves would look like:

save_image-4

Sure enough, on assembly this is exactly how the shelves looked! And there is plenty of room for my dad to collect more. Happy Father’s Day!

For those interested, here is the final list of materials in case you wish to order a similar shelf from 80/20, and here is a link to the OnShape design so you can view and edit it yourself!

  • 16 12″ aluminum tubes, SKU #9000
  • 36 6″ aluminum tubes, SKU #9000
  • 6 3″ aluminum tubes, SKU #9000
  • 14 18.75″ aluminum tubes with a flange on one side, SKU #9005
  • 6 18.75″ aluminum tubes with a flange on both sides, SKU #9015
  • 20 gray plastic tee-connectors, SKU #9130
  • 6 gray plastic five-way junction connectors, SKU #9180
  • 4 gray plastic three-way corner connectors, SKU #9150
  • 12 gray plastic four-way corner connectors, SKU #9170
  • 6 gray plastic caster wheel receptacles, SKU #9113
  • 6 threaded stem swivel casters, 3/8-16×1.5″, SKU #2299

 

Roller Coaster Tycoon Ride Ratings IRL

Do you remember playing Roller Coaster Tycoon, the famous amusement park simulation game that shattered sales records and that remains one of the most beloved computer games of all time? I do, and I also remember the most important part of building any roller coaster in the game – testing. While it may seem mundane to someone who has never played, testing was how you figured out if your ride was going to make any money. The game would give your ride a score in three categories- excitement, intensity, and nausea. The goal was to maximize excitement, keep intensity reasonable, and keep nausea minimal. Largely this score was determined by the g-forces your ride produced. High g-forces could mean high excitement or it could mean people are too afraid to go on your ride. These ratings each varied from ‘low’ to ‘ultra-extreme’- both being scores you generally wanted to avoid. ‘Medium’ and ‘High’ were the sweet spot (except for nausea of course, which you always wanted ‘low’) and if you started to edge into ‘Very-high’ intensity you would start to see a drop in ridership, and thus revenue.

G-force relates the acceleration produced by something to the gravitational pull of the Earth. Most roller coasters pull at most 5G’s, or 5 times Earth’s gravity. They only do this briefly though- on big hills or tight turns. The Space Shuttle, for example, pulled 3Gs on reentry and sustained them longer – amusement park goers are clearly not astronauts! Big drops, lots of inversions, intense helixes, and lots of air-time (or negative Gs, where you feel like you are floating out of your seat) are what sell big rides. While real coaster designers don’t use the Roller Coaster Tycoon rating system to determine if their ride is any good they surely have the same design philosophy- be exciting, be intense but not too intense, and make sure the poor teenagers running the thing aren’t scrubbing vomit off the seat every time people get off. I hypothesized most rides, if they were in the game, would probably fall in the ‘Medium’ to ‘High’ intensity and excitement scores. Fortunately we now all carry around an accelerometer in our pockets built right into our smart phones so we can find out for ourselves!

smartphonediagram.png
This diagram shows how the accelerometer maps to the G’s measured by the Roller Coaster Test Meter.

The above diagram shows the axes I chose so that your phone could measure acceleration while resting safely in a zipped or sealed pocket while you rode a roller coaster. Vertical Gs are along your phone’s x-axis while lateral G’s are measured along your phone’s z-axis. This assumes that you put your phone into your pocket with your screen facing to your left and top-first, by convention.

The game’s formulas for computing the ratings for each ride were somewhat mysterious until the OpenRCT2 project published their open-source code and formulas. We knew for years that primarily the g-forces the ride produced made up the bulk of the score, and other features like theming, dueling trains, and music among other things also contributed. There are also unique multipliers for each ride that come into play.

I am simply trying to build a toy however that you can turn on, throw in a pocket, and share with your friends so I avoided the design route of asking you a whole survey about the ride’s features before you get on. Instead I went a different route to produce a set of formulas that roughly approximate that in the game regardless of what kind of roller coaster you are on, mystery multiples and all, by comparing the scores of real roller coasters to those in the game. Fortunately this summer I have had access to a roller coaster that was in the game and that I could ride in real life- a ‘boomerang’! These roller coasters are everywhere, as they have a small footprint and low cost that makes them perfect for parks wanting to add a coaster on a small budget. The model in the game is ‘Defibrillator’ and it can be found in the ‘Funtopia’ scenario of the original game.

So, readers, I rode it just for you! Just kidding- I am obsessed with roller coasters and the fact that I needed to ride one to complete this project was no coincidence. I started building a prototype of my app using Ionic and Apache Cordova, which would enable me to release my app for you on either Android or iOS without needing to rewrite any of my code. There are excellent tools for making a fun UI (I tried to keep the colors and theming true to the original game) and you can import great packages for social sharing and interfacing with the accelerometer. I ran my app and saved the base score using the basic formulas from OpenRCT2 with no multiples. I then tested for the scores for ‘Defibrillator’ in the game, computed my multiples empirically to scale my ratings appropriately, and voila! We now get scores we would expect if real coasters were in the game!

Screenshot_20180604-132623.png
My Boomerang test ride with raw (unscaled) scores reported at the bottom.

Additionally I wanted to provide you with the raw data that went into your scores, just like the game. I used the awesome Chart.js library to plot the vertical and lateral G-forces live for you right on the screen, letting you have a nice plot of the forces experienced on the ride once you’re done:

Screenshot_20180606-103541.png

Here are a few scores from my recent trip to Cedar Point and Kings Island in Ohio:

It is amazing fun- my favorite highlights are the legendary The Beast having an appropriate intensity of ‘Very-high’ and the crowd-favorite Maverick having excitement at ‘Very-high’. I even rode the new Steel Vengeance– just look at those vertical G’s! Simply download the app from Google Play or Apple App Store, insert the phone top-first and screen facing left into a pocket, and hang on tight! Obviously follow any rules about loose articles (they are there for a reason) but generally as long as you have a pocket that can be sealed this is a fun way to rate coasters, plot their g-forces, and brag to your friends about how you pulled 5G’s on Steel Vengeance this summer. Once you hit the ‘end’ button hit ‘share’ and post the scores to social media, then hit ‘clear’ and enter the next coaster’s name before going and conquering it. Have fun and make good choices!

Download on Google Play

Download on the Apple App Store

Use Python to collect and save data from microcontrollers to your PC

I’m at PyCon 2018 in Cleveland this upcoming week, so I thought that it would be a great time to post a quick Python writeup on how you can collect data from microcontrollers like Arduino and then save the data to your PC to process however you wish! PySerial makes this remarkably easy. This is a great project for science teachers as they show how you can use simple, ~$10 microcontrollers to create excellent data collection systems by connecting a few sensors and modifying my simple software! If you want to skip the tutorial you can download my software for Windows or Mac at the bottom of this page or use the link there to clone my repository on GitHub.

As usual,

THE PROBLEM

How can I collect a bunch of sensor data and then save it to my computer so that I can analyze it in a Jupyter notebook or in Excel?

THE SOLUTION

Over serial, and then use a simple Python program to save the data into a format like CSV that can be easily read by any analysis program!

For this writeup, I was specifically trying to collect data from an analog temperature sensor (TMP36) and then plot that temperature over time. I used a Teensy LC- a $12 Arduino-compatible microcontroller board that is so cheap and easy to use that I throw them at any of my rapid-prototyping problems. It features an ARM Cortex-M0+ processor, lots of analog and digital pins (including capacitive touch!) and a convenient micro-USB port for setting up an easy USB serial connection.

Here’s how I wired up the sensor:

temp_diagram.001
This probably would not pass UL certification.

Since all of my projects must include 3D printing, I also 3D printed a great case off of Thingiverse by a user named Kasm that nicely contains the project and makes sure I don’t accidentally short any of the pins to anything:

20180504_204414
Slightly more safe and professional looking. The temperature sensor is down on the bottom left.

For those curious how this works, the temperature sensor is an analog component that, when given 3-5V, returns a voltage that is proportional to the temperature around it. We are feeding the voltage into an analog to digital converter (ADC) so that our digital microcontroller can make sense of the reading. This means that in our code, we need to convert a number from 0-1023 back into a voltage so that we can use the formula from the data sheet to convert to a temperature. Fortunately, this is a simple conversion:

gif.latex

This will return a voltage in millivolts. The 3300 comes from the 3.3V reference voltage of the Teensy, and the 1024 comes from the Teensy having a 10-bit ADC, meaning it has 210 discrete analog levels.

We can then convert to a temperature using the formula from the device data sheet:

gif.latex-2

This will return a temperature in Celsius. I have adapted the Arduino sketch from this excellent Adafruit tutorial so that it will work with the Teensy:

Note what we are doing here:

  1. We are opening a serial port at 9600 baud (bits per second)
  2. We are reading from analog pin 9 on the Teensy, where the voltage out pin is connected
  3. We are converting the reading there to a voltage.
  4. We are converting that voltage to a temperature
  5. We are printing that line to serial so that it is transmitted
  6. We convert again from Celsius to Fahrenheit
  7. We print this line to serial as well
  8. We print a new line character so each row only has the one pair of readings
  9. We wait one minute to take another reading

Pretty simple! You can change the delay to any interval you wish to collect more or less data- the units are milliseconds. I wanted to only collect data once per minute, so I set it to 60000 milliseconds. Go ahead and flash this to your Teensy or Arduino using the Arduino IDE, making sure your device type is set to Serial. This is important, as without doing it you will not be able to read the data being sent over serial USB.

Entypo_e757(0)_1024.png
All software must have a nice icon.

Now to build the Python program. If you would rather simply download the binaries or clone the repository on Github for the Python client the links are available at the bottom of this page. We are going to use PySerial. We are also going to use Python 3, so make sure you have it installed if you are still using Python 2. Pip install PySerial by typing pip install pyserial. The logger is going to be wrapped in a simple Tkinter GUI. We want to accomplish the following:

  • Read the temperature data being printed to serial by the Teensy
  • Save it to a CSV value
  • Let the user select an interval to read from serial
  • Let the user select where they want the csv file saved
  • Let the user select what serial port they want to read off of.

The end result will look like this:

gui_screenshot
Who says Tkinter has to look ugly?

Let’s take a look at the code and then break it down:

What’s happening here is actually more simple than it seems! Let’s go through how it works, step by step, starting in Main():

  1. We create the root window where all of our widgets are stored (buttons, fields, etc.)
  2. We give it a name
  3. We grid a label
  4. We grid a dropdown menu
  5. We use PySerial to look at all serial ports open on the system and look for one called ‘USB Serial’. By default when you flash the Arduino or Teensy with the sketch shown above it will appear as ‘USB Serial’ on either Windows or Mac computers. We want to make sure the user selects the right serial port, so we filter the choices to these. More than likely there will be only one- the Teensy or Arduino connected to the machine.
  6. We populate the dropdown menu with these serial ports.
  7. We add a button and a counter. The default value of this counter is 60. This is letting the user know that they don’t have to sample every 60 seconds, and they can increment this value if they wish. You can change this value to whatever sampling rate you have set on your Teensy or Arduino in the sketch above.
  8. We add a button that lets the user start the data collection. This triggers a function where we ask the user where they want to save their output file in a save dialog, then open a serial port we called sp using PySerial. We must make sure we open it at 9600 baud, just like the serial port we opened on the Teensy! This is so the two ports can talk to each other. We then create a new thread to run our code in- if we did not, the program would freeze and you would not be able to stop the program because it would constantly be running the timer() function, collecting data, and never returning to the thread running the GUI! We set up this thread to run the timer() function as long as the global variable running is set to true. This lets us have a stop condition to terminate the thread when we are finished. In the timer function we use the filename the user has given to use and every sixty seconds save a time stamp in the first column, the temperature in Celsius in the second column, and the temperature in Fahrenheit in the third column. We read a line of of serial using the PySerial serial port’s readline() function and then convert whatever it gets to UTF-8 to make sure that what we are saving are readable Unicode characters. The output database is always being updated with each minute’s data in a new row! We open the CSV value in append mode (open(f.name, a)) so that we do not overwrite any existing data.
  9. We add a button to stop data collection. This sets the global variable running to false, causing the thread where we are saving the temperature data to stop. This halts all data collection.

And we are done! PySerial handles the heavy lifting and we have a nice, simple GUI to collect our data from. The CSV files can then be read into your favorite analysis program to create plots or fits:

Screen Shot 2018-05-05 at 12.28.30 PM
A quick plot I created in Numbers, showing that over the short interval tested the temperature did not stray far from 23 degrees.

SHARING THE SOLUTION

You can clone or fork the repository that contains all of these files here or you can download binaries to use right away on your system:

Clone the Repository

Download for macOS High Sierra or Windows 10

Stop the flu with this DIY Ultraviolet-C iPad Sanitizer and Charger

THE PROBLEM

Our touchscreens are disgusting, and researchers are increasingly turning to UV-C light to disinfect them. UV-C is naturally blocked by the ozone layer, but can be artificially created and it is known to be germicidal. How can we create a convenient way for people such as teachers to disinfect many screens at once?

THE SOLUTION

Using hardware store materials (making them easy for anyone to procure) we can create an enclosed system that bathes screens in UV-C light and charges the devices at the same time! All we need is a box, a UV-C bulb, a power strip, and some elbow grease. Watch the video above for all the details, and click these links to buy the materials I used to create my sanitizer/charger:

Beat the flu with this easy and inexpensive charger and sanitizer!

20180304_171151.jpg

I also designed this 3D printable iPad stand to sit inside the system. As always, you can download it on Thingiverse or edit it for yourself on OnShape!

 

Exploring 3D Printing with First Graders and Tinkercad

This month I have had the privilege to work with a class of talented first graders in order to teach them about computer aided design, or CAD. This is an essential tool in any engineer’s toolbox, but it using it can get very complicated very quickly. Frequent readers of this blog likely know my fondness for OnShape and it’s spiritual predecessor and industry titan, SolidWorks. These programs are far too complicated for a quick introduction in how to use CAD, and the number of features can be overwhelming. While OnShape does work on iPads, which these children are fortunate to have in their classrooms, there is a much friendlier way to introduce 3D design to children- Tinkercad.

Screen Shot 2018-02-23 at 10.16.35 AM.png
The Tinkercad interface and a quick elephant I drew using the basic primitives it offers.

Users create their models out of simple 3D shapes like boxes, cylinders, and pyramids just like they were building with wooden blocks. The big difference is that these blocks can be morphed and stretched like clay, letting users make pretty much whatever they want. They can set how big they are in millimeters and give a precise orientation in degrees.

Tinkercad is maintained by industry heavyweight Autodesk, which is famous for it’s CAD packages like AutoCAD and 3D art software like Maya and 3DS Max. It gives a nice set of features for beginners to computer aided design- lots of primitives and shape generators to choose from, rulers and workplanes to reference your work to and measure distances, the ability to dimension shapes, and a navigation system straight out of the best parametric modelers are all present here. This is great because it means the skills you develop learning Tinkercad transfer into using other popular packages, like OnShape or Inventor, likely the most similar Autodesk offering. As an engineer this is the biggest selling point for teaching children with this package- I know that whatever they get out of this activity, they can transfer forward to bigger and better things. This cannot be said for other programs I tried- Autodesk has another offering called 123D Sculpt that simulates making something out of clay that I found too limiting and too application specific. Microsoft Paint 3D is an impressive upgrade of the drawing program I remember, but it is more oriented towards 3D art than serious 3D part design.

That said, the activity we tackled was not one of designing parts for a rocket or model car but instead making models of the animals they were studying in their science unit. This may seem like a bad application on the surface for modeling programs like Tinkercad if the goal is to teach students how to make useful parts on 3D printers, but in my opinion it is perfect. A big part of imaging how a part should look or be designed is to imagine how it can be made out of simple shapes and figures. When working with a parametric modeler you often observe a designer create the basic shape out of circles, boxes, arcs, and lines and then begin to add the finer details. Working with something like an animal the children are familiar with is perfect for teaching this higher-level skill. It is one thing to teach them how to draw boxes and spheres and dimension them, but to teach them how to think about what they are designing and introducing them to the creative process is much more valuable than the technical skill of using the program itself. Take this elephant the children drew, for example:

Funky Jaagub-Esboo.png
An impressive elephant the first graders drew. This was the first model they created.

I had shown them my demo elephant, and they used it as a springboard to make their own. They identified the shapes I had used to make mine and impressed me with how they understood the creative process I had gone through to create those shapes. They then extended it by thinking of what shape they wanted to use for standing legs (I modeled mine laying down to make things easier), and what shape they wanted to use for eyes, both of which I did not have in my model! They also creatively added a baseplate for the elephant to be printed on with a label, taking advantage of the Tinkercad text tool. They were already thinking in terms of what shapes they had available to them and how they could be stretched and morphed into the figures they wanted. They were also thinking in terms of what could actually be 3D printed. These students had the privilege of having a 3D printer in the classroom and with a little coaching on my part realized why you cannot have big pieces hanging in thin air if you want a successful print, and identified that parts have to be stacked one on top of the other, not hanging in space. I taught them the ‘sandcastle principle’- imagine that you are making this model in sand. You cannot have sand floating in midair, and it won’t stay up without support. They seemed to quickly grasp the idea and they ran with it.

Here are a few of their amazing creations:

If you would like to view the presentation I gave to the students that explains some easy dos and don’ts about 3D printing and design, I’ve included it here in Google Slides format:

This was an awesome experience, and I know the students enjoyed it too. They are now obsessed with Tinkercad and 3D printing, and I have been amazed by what they have independently been creating. This is truly an awesome tool for introducing 3D art, design, and 3D printing to primary and intermediate students, as there is a lot of flexibility here for making very complicated things. I have even found that for very quick designs Tinkercad is more than adequate and it is fast. I am excited to return and work with a new set of students on another set of animals. I am sure I will be just as surprised at their creativity and inventiveness.

 

Making a Star Wars Poster Lithopane Lamp

Star Wars has some of the most iconic artwork of any movie franchise in history. Nowhere is this more apparent than in its theatrical posters. This lithopone lamp captures all nine main storyline Star Wars posters (with a blank standing in for Episode 9 until the official poster is released!) in a lovely desktop lamp that celebrates your favorite movies. Simply place an approximately 2.5 inch diameter LED puck light (like you would place under a cabinet) inside the indentation in the base and run the cord out the slot!

As always, you can edit this project on OnShape!

Also as usual you can download and print these files right now from my Thingiverse account.