ATU318 – Tactile Text-to-Braille

Play

ATU logo

Your weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist people with disabilities and special needs.

Show notes:
Tactile Text-to-Braille – Bonnie Wang, Team Tactile, Student, Department of Materials Science & Engineering, MIT |teamtactile.com
Charlene Xia (Shah) Mechanical Engineer- Jessica Shi (She) Systems Integrator – Tactile
Matt King to give TEDMed talk: http://tedmed.com/speakers/show?id=687757

App: Find my Friends | www.BridgingApps.org
——————————
If you have an AT question, leave us a voice mail at: 317-721-7124 or email tech@eastersealscrossroads.org
Check out our web site: http://www.eastersealstech.com
Follow us on Twitter: @INDATAproject
Like us on Facebook: www.Facebook.com/INDATA

——-transcript follows ——

 

JESSICA SHI:  Hi, this is Jessica, and I’m the assistance integrator of Tactile.

CHARLENE XIA:  Hi, this is Charlene, and I’m the mechanical engineer of Tactile, and this is your Assistive Technology Update.

WADE WINGLER:  Hi, this is Wade Wingler with the INDATA Project at Easter Seals crossroads in Indiana with your Assistive Technology Update, a weekly dose of information that keeps you up-to-date on the latest developments in the field of technology designed to assist people with disabilities and special needs.

Welcome to episode number 319 of Assistive Technology Update. It’s scheduled to be released on July 7, 2017.

Please check out our website at www.eastersealstech.com, sent us a note on Twitter at INDATA Project, or give us a call on our listener line at 317-721-7124.

***

Technology, Engineering, and Design, or TED. Almost everybody has heard – in fact, almost 2 billion people have watched a TED talk. They are something I am fascinated with and think they are great. If you are not busy, in November 2017, November 1-3, if you are in Palm Springs, California, you might be attending TED Med 2017, which is the healthcare and medicine addition of the world-famous Ted conferences. They are all about ideas worth spreading. The reason I’m excited about TED Med happening in Palm Springs this fall is the fact that Matt King, who is Facebook’s first blind accessibility engineer, is going to be giving a talk about inclusive technology and breaking down barriers to social connectivity. We were so excited when Matt was on our show about a year ago on episode 259. We spent the entire show talking about Facebook accessibility. I would encourage you to be in Palm Springs in November to check out Matt’s onstage TED Med talk. If you can’t, check out our interview with him which is episode 259 of Assistive Technology Update. I’ll pop a link in the show notes. Matt, congratulations. We know you’ll do a great job.

***

Each week, one of our partners tells us what’s happening in the ever-changing world of apps, so here’s an app worth mentioning.

AMY BARRY:  This is Amy Barry with BridgingApps and this is an app worth mentioning.

This week I’m sharing an app called Find My Friends. Great for teens and adults of all abilities, Find My Friends is a tracker app to locate friends and family using an iOS device. Find My Friends allows users to view other users on a map, communicate with them, and receive alerts when your loved ones leave and arrive at a location. For individuals who wonder, the Find My Friends app can provide peace of mind to parents and caregivers. This is also a handy app for young adults with disabilities who are transitioning from high school to college or work. While gaining independence, parents and caregivers can check their whereabouts for ensure safety. When the app is used in social settings and events, the app can reduce anxiety and stress by helping friends find each other. Find My Friends also release the stress of caregivers of older adults with Alzheimer’s or dementia. Knowing they are in a safe location or being able to find them quickly if they become lost makes of this tracking app an invaluable resource.

If used with the Apple Watch, the additional worry of the individual losing their device is lessened. Find My Friends is an iCloud service, so to use it, you will need to be running the latest version of iOS and have iCloud switched to “on” on your phone.

We trialed the Find My Friends app with a typically developing teen. Both the parents and teens found the app to be extremely useful. The parents felt like it was a great way to allow independence, establish trust, and ensure safety.

BridgingApps highly recommends the Find My Friends app. Find My Friends is available for free at the iTunes Store and is compatible with iOS devices. For more information on this app and others like it, visit BridgingApps.org.

***

WADE WINGLER:  I heard about a group of MIT students – actually now recent graduates – who are winners of a hack-a-thon, winners of the Lemelson MIT Award. I’m not exactly sure what that award is but we are going to ask them. They are part of a group called Team Tactile who has done something that has quite blown my mind as it relates to text and braille. I’ve been dealing with text and OCR and braille issues for my entire career, and I think something brand-new and exciting is happening here.

You can imagine how excited I was when Charlene Xia, who is a mechanical engineer, and Jessica Xi, who is a system integrator, working on this tactile project agreed to come on our show via the Internet and tell us about what’s going on. Before we jump into the interview, Charlene and Jessica, welcome and thank you for being on our show.

CHARLENE XIA:  We are happy to be here.

JESSICA SHI:  Thanks for having us.

WADE WINGLER:  You are part of a team of recent MIT graduates. Is that right?

JESSICA SHI:  Yes.

WADE WINGLER:  Tell me a little bit about your team, the folks who are on it and some of the commonalities and maybe how you got together.

JESSICA SHI:  We started this project around a year ago during our junior year. Everyone on the team – there are six of us total – we’ve been friends since freshman year. We all come from different majors. We have people from mechanical engineers, material science, computer science, and electrical engineering. We started out as a really good friends, and that’s what led to this project in the first place. We saw an advertising for this hack-a-thon and wanted to do something fun as friends for this. That’s how we formed the team initially.

WADE WINGLER:  Tell me a little bit about tactile. We are going to get into some technical details, but tell me about where the idea for this technology came from. It was inspired by the hack-a-thon cop but where did the idea itself come from?

CHARLENE XIA:  It was the night before the hack-a-thon. We were all sitting in my room discussing the idea. You are typically not allowed to work on these things physically before the hack-a-thon starts, so we were just throwing ideas around. We started out with a dancing robot, and then we immediately eliminated the idea. We were looking up a concept design online and came across a design for a braille watch. That was pretty awesome. Before we saw the concept, we didn’t really think about making something for the visually impaired community. Once we saw that concept, how about we make something more of a generalized idea like a text to braille converter.

Once we suggested the idea, everyone got excited. This is something meaningful and something we could do within the time period of the hack-a-thon. After we do about it, it was like let’s do a quick Google search to see if anything is out there that does this. A quick Google search turns out there was nothing that converts text to braille directly. We thought we might make it for the hack-a-thon. That’s how the whole journey started.

WADE WINGLER:  My mind is blown already because as someone who has been working in the field for 25 years, for someone to say let’s just make this for the hack-a-thon is pretty bold and interesting, and the fact that you are pulling it off is even more so.

I think a lot of my people in the audience know why text to braille conversion is important. But for those who don’t, tell me about why we want a technology like this.

JESSICA SHI:  I think it’s really important, especially in the information age we are living in right now, to be able to access a lot of information quickly and easily. There are devices out there that can do electronic documents. These days there is still a lot of information that you can’t do that with like menus and other things. We also get this question from people a lot:  why not just text to audio?  That seems to bypass the whole braille part. We think braille literacy is really important. It’s a form of independence, especially for learning things. It’s so hard to learn purely through audio. Braille is really empowering for that.

CHARLENE XIA:  We are students here at MIT, so we go through a lot of physics books, math books. Then we start thinking, how is it possible to learn math through audio?  We were thinking about the equations that people have to read out loud. I can’t conceive of a way to learn math without some kind of printed document.

WADE WINGLER:  I’ve got a good friend who is a stockbroker who is a braille user. He’s totally blind as well. He’s looking at spreadsheet with columns of big numbers all the time. He told me one time, when you are looking at a whole bunch of zeros in a row, knowing exactly where the commas and the periods are becomes critically important. I also am a big supporter of braille literacy and can’t disagree that that’s important.

CHARLENE XIA:  We knew right away that braille is essential. We started looking at statistics, the availability of a braille textbook, the availability of braille devices, and the availability of any braille printed document. The statistic we found wasn’t a very encouraging. The braille devices are very expensive. Braille books are very expensive. Not only are they very expensive, they take a huge amount of space. The number of documents that are translated to a braille version is incredibly low. We saw a potential need for our product.

WADE WINGLER:  Anyone in your team have experienced directly with braille prior to this?

JESSICA SHI:  One of the members of our team had volunteered for an organization that worked at a blind school in Delhi, India. She was doing more of the app development part, but she actually got a chance to visit a school and interact with students. She saw firsthand how students interact with technology, how excited they are with new devices. She was a motivating factor to why we went with this idea in the hack-a-thon. She shared that experience. After the hack-a-thon, we didn’t plan on continuing this project because we didn’t know the market well enough. We didn’t know there was a need. We eventually got in contact with our mentor, Paul. He works here at MIT as a community director. He lost his vision when he was three. Talking to him and listening to his experiences was really powerful. Hearing about the technology he uses and what he likes and doesn’t like about them, and supporting us for why our device is needed gave us a next step to continue this.

WADE WINGLER:  Let’s talk about the device itself a little bit. Give me some description of how it works, how the recognition of text works, how the braille actuation works. Let’s get nitty-gritty with this.

CHARLENE XIA:  The device we imagine right now would be the size of your phone. Imagine the iPhone seven, a little bit larger than that. Ideally it would be flat and portable. On the top of the device would be the refreshable braille display. On the bottom of the device would be the scanning system. What you would do is simply pull out tactile, place it on a handout, a menu, flyer, or any block of printed text, scan the text, and the OCR software will convert the scanned image into digital text and translate it into braille and have it displayed on the refreshable braille display on top.

WADE WINGLER:  Lots of questions then. Let’s talk about the braille side. How does the braille actuate?  Is a traditional piezoelectric?  Is it something new and different?

JESSICA SHI:  That’s something when we first started the project, we were looking at the technology. One thing we know is piezoelectric is pretty expensive, also bulky. We really wanted to develop a new actuation mechanism that’s more compact, which opens the possibility of having multiline braille display and also inexpensive. This is something we wanted to update, introduce new innovation into the refreshable braille.

WADE WINGLER:  Are you to the point where you have workable prototypes at this point in terms of the braille and the entire device?  How far along are you?

CHARLENE XIA:  We do have a working prototype. There is definitely some nitty-gritty stuff we have to tweak. The size of the braille is not quite to the size of the National Braille Press regulation, so we are still a little bit bigger than them and are trying right now to scale it down to the size requirement that’s set up by the National Braille Press.

JESSICA SHI:  Right now we have a standalone battery-powered device that can do image recognition and display in braille. We are working on scaling down the display. The display part is critical, especially when using a new mechanism that is not traditional. We are basically putting a lot of our energy into that.

WADE WINGLER:  That’s an issue that the industry is working on trying to crack. They call it the “Holy Braille”, to get the one that is less expensive and crisp and still does a good job. I think you are on the verge of breakthrough in that area. Talk to me about the input side. We have a camera that is doing OCR. It has to be doing a buffer so there is text in there. As a real-time life conversion?  Is there a buffer where you can scroll back and forward?

CHARLENE XIA:  It is Bluetooth capable, see can configure it when sitting on your phone or if you have an email on your phone. It’s not just for completely printed text. We also want the flexibility of the electronic document. The user has options. For example, when you read a newspaper, the format is different because it is column based. If you scan one section, it doesn’t quite translate the entire – you need to scan the entire page in order to have a proper translation because it’s a weird structure. The OCR will provide feedback to the user to let them know whether it is recommended for them to scan the entire section before reading the resulting translation or is it better for them to – this is okay to scan section by section. For example if you are reading a novel, a novel is pretty standard. It goes by paragraph down the page. This way the user can scan section by section. We have memories stored on the device. Once you scan a section, it scans it so you don’t have to scan one section, realize you have to go back and look at a previous section and scan it again. There is a [inaudible] that the user can pan back and forth between each section.

WADE WINGLER:  That make sense to me for reading materials in the immediate. Are you also going to have longer term storage see can have a chapter of the book or something in there?  What would that look like?

JESSICA SHI:  We wanted to have local storage on the device itself, but what we really want is to have you be able to store it on your phone via an application. We learned that a lot of people have smartphones with them. Smartphones are great because they do have voiceover and all these capabilities, and it’s basically a minicomputer. We are hoping that in terms of large books or more long-term storage, you will be able to have it stored on your phone via the app and pull it up that way.

WADE WINGLER:  And possibly cloud storage as well.

JESSICA SHI:  Yeah.

WADE WINGLER:  Talk to me about the accuracy of your OCR. How does it handle things that are nontext elements that they come across?

JESSICA SHI:  If the text is not perfectly aligned, as long as the angle is within 15 degrees, it can still detect text. The image quality is okay. It is still able to extract text. In terms of images, this OCR is something we are working on but is also more of an open sourced network. There is a lot of machine learning that is going into this. Right now, a capability it can do is if it detects images, rather than just skipping over it, is it able to actually describe this image with the image processing that is going on for pictures.

WADE WINGLER:  It sounds like there is a great opportunity to integrate a lot of features into this device. I can see why you are excited about it. Another technical question:  what is battery life going to look like?  You are doing a lot with a small device. I know braille actuation is going to be part of the secret with that. What is your target for battery life?

CHARLENE XIA:  We are targeting six hours. From the users we talk to, reading braille, using your finger, the longer you read, the finger sensitivity decreases. We found the optimal time is if you read continuously for six hours, your finger is going to feel slightly off. Six hours is our minimal time. Ideally we want to go a full day.

WADE WINGLER:  But right now you are thinking you may need to recharge your finger before you recharge tactile?

CHARLENE XIA:  Yes.

WADE WINGLER:  Some questions that aren’t so technical. In the preinterview, we chatted that the engineering students seem to be spending a lot of time with PR and media. Tell me about some of the media attention you have gotten for this, because that’s how I found you.

CHARLENE XIA:  It’s definitely something we never prepared ourselves for. I have to say it’s all been very exciting. Talking to reporters, talking to you, it gives us hope and motivation that people are looking forward to this device and the people are excited about this device. That’s what drives our passion, to keep working on it. I know from your excitement, that’s how we get excited.

WADE WINGLER:  I know that has to be affirming. You won the hack-a-thon?

JESSICA SHI:  Yes.

WADE WINGLER:  And you’ve won a thing called the Lemelson Award?  I don’t know what that is. Can you tell me about it?

CHARLENE XIA:  Lemelson MIT is a national competition that celebrates inventors. They celebrate a graduate inventor and undergraduate team inventors. We submitted tactile this year in late January, early February for the prize. Actually earlier. They announced the interview –

JESSICA SHI:  In April. After the announcement, one of the things you get as a winner for the Lemelson MIT student prize is this national campaign that gets you exposure, which is very valuable. Especially for us working on assistive technology, it amazing to spread the word and to get the community excited about what we are working on.

CHARLENE XIA:  A little bit of history on Lemelson. Lemelson MIT, this competition, is funded by the Lemelson Foundation. The foundation was started by a wonderful guy, Jerry Lemelson, who had over 600 patents under his name. He’s basically this amazing inventor who would stay on his drawing board and thinking up those amazing ideas. The Lemelson Foundation wanted to celebrate inventors, so they had a competition to celebrate undergrad/graduate inventor teams.

WADE WINGLER:  That’s fascinating and that’s new to me. I read that you are working on patent work with Microsoft as well. Is that related to the Lemelson Award or is that separate?

JESSICA SHI:  That’s separate. The program with Microsoft is they are celebrating female inventors. There are some statistics that on all patents, less than 10 percent of them have a female inventor named on them. They wanted to promote STEM education for young women. Because our team is all-female, they thought it was a great fit for the program. They are basically sponsoring us to get a patent on a device.

WADE WINGLER:  That’s great. We are getting close on time for the end of the interview, but I have a question that I know my audience wants me to ask. The when, where, and how much kind of questions. What are you thinking in terms of availability and price point?

CHARLENE XIA:  We would like for Tactile to be ready for manufacturing within a year. To get it ready for manufacturing, we had to go through design challenges, we had to go through a lot of user interviews, and durability to see long-term. You use it one day and the device is cool and it works well, but using it for six months all of a sudden something broke. We want to make sure to fix that for the manufacturing stage. One year is our goal for the tactile to be manufacture ready.

WADE WINGLER:  Based on our recording time, we’re looking at Summer 2018, right?

CHARLENE XIA:  Yes. Summer of 2018. For the target price, we would like to keep tactile under $500.

WADE WINGLER:  That’s impressive. You have some work to do. Like I said, I think people in the industry are paying attention to this because you have some major innovations and hurdles to clear here.

CHARLENE XIA:  We set the strict goal for ourselves because we want to work hard to achieve it.

WADE WINGLER:  That makes total sense. As you think about the future, we talked about the functionality that you are building into it, do you have longer-range plans for functionality you would like to see in the device?

CHARLENE XIA:  We want to make sure our device is modular. We also throw ideas around like, besides a multiline, is it possible to do a full page?  If it’s possible to do full page, is it possible to include input, so make a braille tablet like an iPad for the visually impaired community?  We throw ideas around that we would like to attempt as we trial new designs.

WADE WINGLER:  Based on what you’ve been able to do so far, I have confidence you are going to be able to do those and additional amazing things related to it.

I know people are going to want to follow your journey. After hearing the interview today, they will want to reach out to you and know when, where, and those questions. What contact information would you like to provide so people can follow your story and reach out to you?

JESSICA SHI:  If you want to stay up-to-date with our development and learn a bit more, you can go to our website TeamTactile.com. Or if you would like to directly reach out to the team, share some thoughts, you can email us, tactile@MIT.edu.

WADE WINGLER:  Charlene Xia is a mechanical engineer, Jessica Xi works as a system integrator, they are part of team tactile working on what promises to be a revolutionary text to braille device. Thank you so much for being on our show today.

JESSICA SHI:  Thank you. It was a pleasure.

 

 

 

 

 

WADE WINGLER:  Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on Assistive Technology Update? Call our listener line at 317-721-7124, shoot us a note on Twitter @INDATAProject, or check us out on Facebook. Looking for a transcript or show notes from today’s show? Head on over to www.EasterSealstech.com. Assistive Technology Update is a proud member of the Accessibility Channel. Find more shows like this plus much more over at AccessibilityChannel.com. That was your Assistance Technology Update. I’m Wade Wingler with the INDATA Project at Easter Seals Crossroads in Indiana.

***Transcript provided by TJ Cortopassi.  For requests and inquiries, contact tjcortopassi@gmail.com***