InGlove

Curt, Shruthi, Vedant

Weekly Accomplishments

Curt –

I am going to present this in a different tone relative to previous posts that I have written. Specifically, I am going to allow more digression / stream of conscious comments in order to capture some of the context of last week (hopefully for entertainment value). If you don’t care for the longer version I have included a summary.

(Summary)

• 3D printed all prototype finger parts
• Constructed first version of the finger. Noting all construction issues.
• Developing electronics of the finger (and struggling with a servo jitter issue along with sporadic movements)
• Figured out the erratic servo issues are due to the ESP32 being a 3.3V microcontroller that has an issue controlling 5V servos once the Vcc voltage drops (motors starting moving). Current solution to this is to use a different 5V microcontroller until I get a 5V level shifter. Also servos will be driven off a separate USB powersupply.
• Programmed a servo driver interface to determine appropriate starting joint values and joint limits. This will hopefully be shown in the demo.
• I wear the finger to understand how it feels (weird) and
• Ordered a glove that I like (also some through-hole PCBs for soldering later in the project).
• Goal for next week is to crank out the flex sensors and integrate everything together.

(Long Version)

My work this week focused on finishing the prototype robotic finger and its mounting in order to start wearing the device thereby gathering insight into improvements for future versions. Below are several pictures from early in the week as I started exploring the integration process. During this time I was also printing parts at both the 3D printer on the second floor and the 3D printer at College Library InfoLab.

After picking up the 3D parts on Friday (in the much needed break I had during the day of the UIST deadline) I was able to get started on constructing the full finger.  More specifically, I started construction of the finger after a quick break to decompress from submitting the paper I was coauthoring (where I had a night’s deficit of sleep and fresh dose of caffeine mixing with the adrenaline already keeping me awake). I don’t have a picture of this version as it never took complete form. Instead I found I had to modify the components in order to get the individual pieces to work together. Unfortunately this meant going through all of the extra hinges I printed (in anticipation of this). Furthermore, I ended up cracking and warping most pieces (pretty sure this was not the sleep deprivation just poor part design).  In the end, I did manage to create a finger that came together as a complete piece (along with plenty of notes on what to change for future versions). Below is a picture of the finger in all its glory (note it is actually a later picture once it started working).

On Saturday I resolved to complete the finger for the demo on Monday. In comparison to my induced manic state on Friday (solely as a response to the unhealthy cocktail of stress, sleep deprivation, and caffeine), I was less productive. On Saturday, I was able to wire up the finger and started programmatic control of the servos. During this time I found an issue with using the ESP32 as my microcontroller, in that the ESP32 is a 3.3V device whereas the servos are ~5.5V. Ideally this is not a problem since the 3.3V should be barely high enough to be detected as high but once the servo starts to move (thereby drawing current) the voltage rail sags causing the microcontroller (connected to the same USB hub as servo motor voltage regulator) IO voltage to sag as well. This manifests as unpleasant jittering with the occasional sporadic spasm (which made me question whether the thing I was created was even safe to wear nevertheless I continued on). I ended up banishing the finger to a plastic box so that it would not make a freedom-focused break for the edge of the desk while I was debugging.  Saturday ends with me using an old arduino-clone microcontroller that is 5V to get rid of some of the odd behavior. I am planning on getting a 3.3V to 5V level shifter IC which should work even when the voltage rail sags.

Sunday, I finish some debugging on the servo control using the arduino clone and then use the serial interface I wrote to capture the joint information that I think is acceptable for the finger. After this I placed the finger onto my hand to get a feel for the device. As a short description, “it feels very weird when the servo moves”. Though this was nothing like an earlier moment where I only had the base on and reached into a cabinet getting caught on an edge that frankly I did not notice before. This event caused a moment of panic as my body had not adopted the base as part of its mapping. Lastly on Sunday, I ordered the glove that I intend to use for the final version. Final figure is an action shot with the finger.

My goals for this next week is to develop all of the flex sensors. I purchased the electrical tape I need (and so long as the Velostat, conductive thread, and conductive fabric last), I should be able to complete all eight sensors. Then it is onto implementation of the algorithm. I did put some further thought into this and I will be using PCA in order to make sense of all the data entering from flex sensors in addition to the orientation and acceleration vectors captured for the hand. From the PCA output I will construct a supervised learning ML task, starting with a linear regression (we can all dream that the solution is simple), but moving to something more complex like nonlinear regression, random forest regression, etc. Specifically to make it a supervised learning task I will create a program that requests a set of gesture and then I will enter the servo joint values associated with that state.

Shruthi –

For this week I worked on building an android application. I spent sometime simply understanding the basics of android app development process. After that I tried reading through the particle documentation of its API and was trying it out. My goal was to have an app read the data off a particle photon. However, the particle documentation is a bit vague and assumed a lot of knowledge of android development and I found it hard to build something from that documentation alone. I then started looking around for examples and eventually found one. In the process I realized that in order to establish connection with the particle server I need to use sockets to establish an open ssl connection using REST interface. To set this up I installed all the binaries required to setup curl on windows, however it is currently giving me an error when I try to POST a request. I plan on spending time maybe until Wednesday on this before trying alternate approaches.

Vedant – 

This week we looked more into developing an app that can relay information from the Particle Photon to the Smart Switch/Bulb. I looked into the API for the smart switch app and how to obtain information about it such as the device ID, local key and IP address. I worked on making 5 more of the DIY flex sensors which we will sew on to the glove. Additionally, I started looking into how to send IR signals using a photon and converting the IR codes into the format the photon can use.

Materials Lists

Home Assistant Sub-Project

1. Particle Photon – $19.00 (1)
2. Flex sensor – 4.5 inches – $12.95 (1)
3. Flex sensor – 2.5 inches – $7.95 (1)

*We decided to go ahead with more of the DIY flex sensors. So we might possibly need more velostat and copper sheets and conducting thread.

SRF Sub-Project

Already purchased / owned

1. Glove for prototype [final version subject to change based on prototype]
2. Sparkfun IMU – $14.95 (1)
3. Flex Sensor – 4.5 inches  – $15.95 (1)
4. ESP32 Dev Board – $15.00 (1)
5. High Torque Micro Servo – $9.95 (3)
6. Resistive Force Sensor – $7.00 (1)
7. Flex sensor – 4.5 inches – $12.95 (1)
8. 3D printed  SRF – ~$9.00 (N/A)

Need to Purchase / Being Shipped

1. Resistive Force Sensor – $7.00 (5)
2. Flex sensor – 4.5 inches – $12.95 (3) and/or Flex Sensor 2.5 inches – $7.95 (8)
3. Glove [Final design] – $14.99 (1 ordered)
4. PCBs – N/A (N/A ordered)
5. 3.3V to 5.0V Level Shifter IC – ~$1.00 ( 1)

Areas of Concern

Curt –

Falling behind on the glove portion. Need to just build all of the sensors and get them mounted on the glove along with the IMU. Also, I need to work through the issues with the finger that the prototype demonstrates. Some of these may not be addressed by the showcase as I would need to test alternate approaches to find the best. One important note in particular, is that I am going to need another servo in order to get a 4th degree of freedom.

Shruthi –

The particle photon community is not very extensive and hence debugging has become quite challenging as you run into new errors.

Vedant –

I was able to get the device ID and IP address but I am having trouble getting the local key for the smart switch. I did some more research and found couple of other ways to get it, so I will be working on that next week. Additionally, I was having trouble finding a IR code converter (from HEX to raw) for the IR codes for my TV, so I’ll have to look a little more into that as well.