ATU209 – Alt_Text_Bot for Twitter (Cameron Cundiff), ICanConnect program extended by FCC, Man drinks beer with brain controlled robot arm, Open Source assistive technology hardware, iOS hidden low light mode

Play

ATU logo

Your weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist people with disabilities and special needs.

Alt_Text_Bot – Cameron Cundiff

FCC extends tech program for deaf-blind http://buff.ly/1cjnf1X

Brain implant controls robotic arm – with the power of thought http://buff.ly/1cjmplM

There’s a hidden ‘low light’ mode buried deep in your iPhone’s settings http://buff.ly/1LJPxPX

Using open hardware to build better assistive technology http://buff.ly/1RodgbB

App: Storybook maker www.BridgingApps.org

——————————

Listen 24/7 at www.AssistiveTechnologyRadio.com

If you have an AT question, leave us a voice mail at: 317-721-7124 or email tech@eastersealscrossroads.org

Check out our web site: http://www.eastersealstech.com

Follow us on Twitter: @INDATAproject

Like us on Facebook: www.Facebook.com/INDATA

 

——-transcript follows ——

>> Hi, this is Cameron Cundiff.  I’m a developer and accessibility advocate at Thoughtbot.  And this is your Assistive Technology Update.

[Music.]

>> WADE WINGLER:  Hi, this is Wade Wingler with the INDATA Project at Easter Seals Crossroads in Indiana with your Assistive Technology Update, a weekly dose of information that keeps you up‑to‑date on the latest developments in the field of technology designed to assist people with disabilities and special needs.

Welcome to Episode No. 209 of Assistive Technology Update.  It’s scheduled to be released on May 29th of 2015.  Today I’m super excited to talk with Cameron Cundiff, who is the creator of Alt Text Bot, a thing that turns Twitter pictures into accessible text.  We talk about the FCC extending the iCanConnect program, a man who has, yep, drunk a beer with a robotic arm implanted into his brain, a way to turn your iPhone into a super low light device, and an argument that open hardware would make for better assistive technology.

We hope you check out our website.  You’ll find that over at www.eastersealstech.com or shoot us a note on Twitter at INDATA Project or call our listener line.  Give us some feedback.  Ask a question.  We love to hear from you.  That telephone number is (317) 721‑7124.

If you like this show, you might like to know that we do a couple of others, as well.  Every week we put out one called Accessibility Minute where we give you quick and easy tips on accessibility and also twice a month we do one called ATFAQ, where we answer your Assistive Technology Frequently Asked Questions.  Check them all out over at accessibilitychannel.com.

The headline from thehill.com reads:  “FCC Extends Tech Program for Deafblind.”  The National Deafblind Equipment Distribution Program, most people know it as iCanConnect, has been running in pilot mode for the last few years.

And the idea was that if the FCC were to distribute telecommunications equipment to people who are deafblind, they would communicate better with the community and get access to resources and all those kind of things.  So for the last couple of years, that has been happening.  Specialized Braille equipment and other magnification and amplification systems have been distributed, and it’s gone very well.

So just very recently, the FCC announced that this program will no longer be running in a pilot mode and is going to be extended.  There’s also some things happening there related to making set top boxes more accessible and making emergency communications more accessible to people who are deafblind, as well.

So in a quick note, in the interest of full disclosure, Easter Seals Crossroads, where we live here at the INDATA Project, actually does provide services in Indiana related to the iCanConnect program.  So because we are tied to that program and do some business for that program, we’re particularly excited to say that the program has been taken out of pilot mode and extended for the future.

So, if you want to read more about this development, check our show notes.  I’ll have a link over to thehill.com and you can read all about this new extension of the iCanConnect program.

This headline in The Guardian surely caught my attention.  It reads “brain implant controls robotic arm with the power of thought.” Erik Sorto is a 34‑year‑old American who has incurred a spinal cord injury around 10 years ago, left him paralyzed from the neck down.

A couple of years ago he underwent a surgery in which some electrodes were implanted into the posterior parietal cortex, the PPC.  And that’s the area of the brain that gives rise to the intention to move.  Apparently, there have been some other neuroprosthetics that had been installed in areas that were designed to control the movement of the body, but this is a new kind of neural implant or a neuroprosthetic implanted in the brain where you think about the things you’re going to do.

And what they’ve done is ‑‑ this has resulted in a situation where he’s able to control a robotic arm.  He’s been able to control it to play rock, paper scissors, to control other kinds of movements around his environment.

And Erik Sorto said he was very happy because now he’s able to drink his own beer without having to ask somebody to give him that help.

So, very interesting.  The article mentions that we’re starting to blur the lines between what is human and what is robotic, and this is a real concrete example of how a neural implant and thought control can be used to modify things in the environment.

So I’m not going to lie.  A little bit creepy, but pretty incredible science happening here.  So, anyway, I am going to pop a link in the show nodes over to The Guardian, and you can read more about how Erik Sorto is using thought control and a robotic arm to drink a beer.  Check our show notes.

I spend a lot of time talking about the accessibility features that are built into IOS, but I got surprised here recently by one that I didn’t know was in there.  Did you know that there is a hidden low light mode that’s buried in the zoom function of your IOS device?

If you go into your settings and go to “general” and then “accessibility,” and under “zoom,” turn on zoom and click the button that says “show controller.” It will then put a little joy stick looking thing on your screen.  And when you tap it, it brings up the controls.  And in those controls, you’ll find zoom in, wind zoom and choose filter.  If you hit “choose filter,” there’s an inverted, a gray scale, a gray scale inverted and a low light filter that you can turn on in there.  And that will give you access to a very, very low light, low contrast version of your IOS screen.

So obviously if you have low vision and light sensitivity, that’s going to be a hidden feature that will be really helpful.  Also if you’re trying to check your text messages in a crowded movie theater, that might also be a helpful option, as well.  So you can turn the brightness down on your screen pretty easily.  And most people know about that.  But this goes even lower into this low light mode that will give you super low lighting on your iPhone screen.

So I’m going to pop a link in the show notes over to a Business Et Cetera article that will tell you more about how to do that, give you some screen shots.  Check our show notes and learn how to turn the lights way down low on your iPhone.

According to his bio on opensource.com, Spencer Hunley is an autistic professional, open source assistive technology enthusiast and advocate for people with disabilities.  He recently posted a story called “Building Better Assistive Technology with Open Hardware.”

In this article, he advocates for less proprietary hardware when it comes to assistive technology.  He kind of points the finger at the augmentative and alternative communication industry.  He talks about how most of those devices are built on proprietary hardware and have lockdown software.  And obviously there are industry reasons for doing that, but he goes into some of the benefits of the idea of creating more open source hardware, so tablets that might be operating on Linux and designed to use assistive technology.

And he kind of outlines some of the projects that have been done like that.  He talks about a project called Sesame, which used an Arduino Uno, a bluetooth module and a servo.

He talked about a project that used an RFID reader and electric door lock to do an RFID door looking system.

And then he even talked about what he calls an intriguing project called Eyewriter, which was developed to use a cheap pair of sunglasses, a PS3 camera, infrared LEDs, some wire and a couple weeks’ worth of development and a home grown eye gaze system was created.

So obviously there’s a lot to this.  Open source is a growing movement, hardware and software both, but it’s an interesting perspective by an assistive technology user about the benefits of assistive technology open source hardware.

So I’m going to pop a link in the show notes over to opensource.com, and you can check out Spencer’s opinion about the importance of open source assistive technology hardware.

Each week one of our partners tells us what’s happening in the ever changing world of apps.  So here’s an app worth mentioning.

>> AMY BARRY:  This is Amy Barry with Bridgingapps.  And this is an app worth mentioning.  This week’s app is called Storybook Maker.

A great summer app, Storybook Maker allows the user to write, illustrate and publish their story.  Using text, photos, graphics and audio recordings, the app allows an adult or child to design their own book.  There are many uses for this app, but one we have found especially useful for children is a method for discussing upcoming changes in a schedule due to vacations or some other event that is out of the ordinary routine.

Using Storybook Maker, a caregiver can write a social story about their vacations or weekends.  Using actual photos, the story can be customized for the user.  Older children and adults can also use Storybook Maker to write stories about the event and then share with family and friends.  It’s a great way for the child to share experiences with their friends and teachers when they return to school after summer break.  It’s also a great way to encourage written language and auditory comprehension.

We have used this app with children between the ages of two and 10 with diagnosis of Down’s Syndrome, speech delay, autism spectrum disorders, and anxiety.

Storybook Maker can be used to design, create and share social stories, picture books, photo collages, scrapbooks and vacation albums.

Your books can be shared in three ways.  You can publish as a PDF document and email it; publish the book as an multimedia book and share it; or publish the book to your public library and share it with all users of Storybook Maker around the world.

Storybook Maker is an excellent tool to encourage writing and story creation.  The app is $2.99 in the iTunes store, and it’s compatible with IOS devices.

For more information on this app and others like it, visit Bridgingapps.org.

>> WADE WINGLER:  Anymore, it’s kind of hard to pay attention to the world of assistive technology without paying attention to the way information flows around the web and the Internet.  And Twitter is one of those tools that I use on a regular basis and I know that I’m certainly not alone in that.

Recently I have seen the thing coming across my radar called Alt Text Bot, which does some interesting things with images and accessibility.  And so based on what I’m seeing, I reached out to the creator of Alt Text Bot, his name is Cameron Cundiff, sorry, Cameron.  And I have him online here today to talk with us.

So, first of all, Cameron, how are you?

>> CAMERON CUNDIFF:  Good.  Hi, Wade.  I’m doing well, how are you?

>> WADE WINGLER:  I’m great.  Thank you so much for being on the show today.  Tell me a little bit about your background and how did you get to the point in your life that something like the accessibility of images on Twitter was something that was important to you?

>> CAMERON CUNDIFF:  Sure.  I have a background in fine arts and studied fine art in undergrad and ended up sort of gradually moving towards technology as a practice and ended up doing an internship at Adobe for accessibility just sort of randomly.

So my background led me in sort of haphazardly to accessibility through my interest in design and through my interest in usability.  And at the time, web standards was really a hot topic.  And accessibility is sort of a poster child of web standards.  So this job that I took on at Adobe was my first foray into accessibility.  And ever since then, I’ve really taken it to heart.

>> WADE WINGLER:  Well, so that’s a great background to have, somebody that comes into it with an artistic background and then somebody who is interested in technology.  It’s an unusual mix of skills and I’ve glad that you have it.

So you made this thing called Alt Text Bot.  Tell me a little bit about what it is and how does it work.

>> CAMERON CUNDIFF:  Sure.  Alt Text Bot adds descriptions to images on Twitter.  The idea behind Alt Text Bot is that Twitter is an important part of our global conversation, and increasingly it’s becoming very image‑heavy.  And there’s a couple reasons for this, but I notice it as a trend and also observe that Twitter doesn’t have any way right now to add alt text images.  And this is sort of at the top of the list in terms of accessibility concerns, is that images should have alt text for folks who are blind using screen readers.

So I figured I could fill this gap using off‑the‑shelf technology and make a difference for people who are blind who are using Twitter.

>> WADE WINGLER:  And we talked about Alt Text Bot.  Are we mostly talking about accessibility for folks that are blind or visually impaired, or are we talking about other folks, as well?

>> CAMERON CUNDIFF:  Mostly people who are blind.

>> WADE WINGLER:  Yeah.  So, tell me a little bit about the user experience and what that’s like.  If somebody who is blind is interested in having images described, what does that look like?

>> CAMERON CUNDIFF:  The first step is if you send a message to Alt Text Bot on Twitter that has an image attached or if you retweet something with an image attached, Alt Text Bot will ask you to sign up.  It’s a quick ‑‑ you add your Twitter handle and email address at AltTextBot.com.  And then it’ll send you a description from then on out for any image that you send it.

So it replies in line to the tweet and gives you sort of a high level description of the image that you sent to it.

>> WADE WINGLER:  Oh, that’s cool.  So it’s as simple as once I have my account set up, I tweet to it or mention it and then it will retweet back a description.

>> CAMERON CUNDIFF:  Correct, yeah.

>> WADE WINGLER:  Okay.  So now I’m fascinated with that.  Who or what is making that happen?  What’s going on behind the scenes?  Because it’s not as simple as say, oo, the whole picture, thousand words thing.  How does it work?

>> CAMERON CUNDIFF:  It works by ‑‑ well, first of all, it uses the Twitter API to listen for any requests from people who are sending it messages.  And it takes those images and those requests and sends it to a third‑party API called the Cloud Site.

Cloud Site is the technology that powers a couple of other applications that are used I think pretty frequently by people who are blind, one of which is called TapTapSee.  And it’s an API that I’m using to translate these images into text.  So once I get a response from the Cloud Set API, I then send the message back through the API on Twitter to the person that requested it originally.  It takes about 15, 20 seconds, that whole process.  But then you get a notification that says, hey, here’s what this image is.

>> WADE WINGLER:  And even at 15 or 20 seconds, that’s remarkably fast for the benefit that comes from this.

>> CAMERON CUNDIFF:  Ah‑huh.

>> WADE WINGLER:  So, I understand there’s this kind of Twitter bot that does this.  And recently you mentioned that there’s something going on with the Safari extension.  Can you tell me about that?

>> CAMERON CUNDIFF:  Yeah, I’ve been exploring using the same tooling with Cloud Site to build browser extensions.  And the goal of the browser extension would be to make this a little bit more of a push technology instead of a pull technology.

So right now you end up having to ask for these descriptions.  But what about, you know, just having them show up automatically, or with a little less work?

And that’s where the idea for extensions came up.  I built a first prototype for a Safari extension.  And it is in progress.  And I’m also working on a Firefox extension.  And these would apply initially to Twitter, but they could also apply to other applications like Facebook, for example.

>> WADE WINGLER:  Well, so that’s fascinating.  And so do you think the infrastructure there is ‑‑ you think the infrastructure is there to try to port this over to other social media platforms, as well?

>> CAMERON CUNDIFF:  Yeah, I think so.  The biggest blocker right now, the biggest challenge with the technology is that the response from the API, the image recognition API, does take time.  It’s not a lot of time.  But 15 seconds when you’re navigating a page does feel like a long time.

So I’m working on ways to, you know, cache images.  I mean, there’s a couple of strategies that I can take to make it a little more snappy.  But I think that it could absolutely apply across the web.

>> WADE WINGLER:  Yeah, that’s fascinating.

So tell me a little bit about the accuracy of the image descriptions.  Does it get it right most of the time?  What does the subjectivity look like in that situation?  And has there been anything, any missed recognitions that made you giggle?

>> CAMERON CUNDIFF:  The image recognition is quite good.  I was surprised.  Image recognition is a very difficult problem in computer science.  And I was really pleased with the results from the Cloud Site API.

Let’s see.  The average response rate?  You know, every 20 images or so you won’t really get a good read on the image.  In those cases, they just kind of say we don’t know ‑‑ the image recognition software couldn’t figure out what this is.  But for the most part it returns descriptions, and those descriptions are, I mean, subjectively speaking, quite good, I think.

>> WADE WINGLER:  Yeah.

>> CAMERON CUNDIFF:  And the tone of the descriptions is also quite neutral.  And that’s interesting to me, is that the experience of being on Twitter is oftentimes very politically or emotionally charged.

So when I’m looking at the Associated Press stream and I see an officer in a line at the riots in Ferguson.  For me, that’s a very emotionally charged image, and I immediately have associations; but Alt Text Bot will say something like, you know, “police officers standing in the street,” which I think is a good thing.  I think it’s good that it allows the reader in this case to get a baseline context and then decide for themselves how they want to interpret that.

>> WADE WINGLER:  Yeah.  Which is kind of exactly how you would expect a robot to interpret something like that.

>> CAMERON CUNDIFF:  Ah‑huh, yeah.

>> WADE WINGLER:  So we’ve talked a little bit about the time delay and how that may kind of impact doing things other than Twitter.  Are there any other limitations or any other thing that you wish worked better about the system the way it currently is?

>> CAMERON CUNDIFF:  There’s a couple of problems or challenges that I’m facing right now.

The first probably most significant challenge is the Twitter API limits.  So over the past two weeks, there have been a couple cases where the Twitter ‑‑ I maxed out the Twitter API, of the amount of requests I can make to Twitter with Alt Text Bot, which basically means that it won’t work for a little while until Twitter decides, based on some rules, that Alt Text Bot can start tweeting again.  And that’s obviously a problem, especially if people are going to be using this as sort of an everyday tool.

And I’ve come up with a couple of strategies for handling that.  The first is to sign up.  So to begin with, the Alt Text Bot was used so that anyone can send it an image and it would respond.  And that got out of control really quickly.  I was getting thousands of requests every day and hitting those API limits by like 3 in the afternoon.

>> WADE WINGLER:  Wow.

>> CAMERON CUNDIFF:  So I put in the signup as sort of like a little bit of work to use it.  The signup is very low overhead.  It’s just two fields and you’re good to go.  But it’s been enough to sort of stem that influx of requests.  At least for the past week or so it’s been enough.

Now, I do foresee that, you know, eventually becoming ‑‑ it continuing to be a problem.  So I’m taking it one step at a time.  It’s been much more successful than I anticipated, and so every day is sort of like responding to new challenges with the Twitter API limits.

>> WADE WINGLER:  I’m chuckling here because folks around my Shop I always say “oh, that’s a good problem to have.  Oh, that’s a good problem to have.”  And they always push back and say “yeah, but it’s still a problem.”

[Laughter]

>> CAMERON CUNDIFF:  Absolutely.  And I’m committed to improving it and making it work.  So I take it seriously.  And I’m going to make sure that it’s useful to people.  So I do work on solving those problems.

And then the other issue is right now ‑‑ so the Alt Text Bot was originally built as part of a hackathon partnership with AT&T and NYU hosted a hackathon here in New York, which is where I live.  And I built Alt Text Bot in a weekend, the initial version and built it on the cloud site API.  And it was great.  And it won first place.  And I won some prize money.  And that was awesome because it meant that I could fund the application’s API usage in the interim.  I mean, it was a significant amount of prize money.

So in the meantime, I’m paying for the cloud site API with the prize money that I won.  And that’s fine.  That’s exactly what it’s there for.

Within the next couple months, I think I’ll probably end up figuring out a way to make this a paid service and having it be sustainable from a financial perspective.  But right now it’s not ‑‑ it hasn’t become a problem; it’s something that’s on my radar.

>> WADE WINGLER:  Yeah, and as you look at the future of Alt Text Bot, I think that’s important because you’re clearly adding value here and asking people to, you know, to contribute to that sort of makes sense.

So what other kind of things are in your crystal ball as you look at the future of Alt Text Bot or other kind of work in this area?  What’s coming down the pike?

>> CAMERON CUNDIFF:  I really like the vision of having this apply across the web.  And, really, the immediate future, that means the extensions.  So I’m going to be doubling down my efforts on a Firefox extension, in particular because I think Firefox with NVDA is one of the better screen reader browser combinations out there.  And I know that ‑‑ I think Internet Explorer is top amongst screen reader users.  And then Firefox comes second.  Firefox is much easier to develop extensions for.  So a combination of meeting halfway.

So building a Firefox extension that will at first work with Twitter.  And then one by one trying to nail down these other hubs of activity and seeing:  Can this apply across the web?  It’s a lofty goal, but I think it’s something I aspire to.

>> WADE WINGLER:  Yeah, it seems to me you have a lot of momentum in that direction, and I’m excited for what you’re doing.

So, Cameron, if people want to learn more or get connected, give us a website address, again, and any other advice you have on how to learn more about what you’re doing.

>> CAMERON CUNDIFF:  Sure.  Well, the Twitter account is alt_text_bot.  And it has a little bit of information in the profile of the Twitter account.

And also you can sign up at AltTextBot.com and there’s a link there with the background on the hackathon and also to the source code for the application.

So just as a side note, anyone can go and see how I built this by going to AltTextBot.com and clicking the link to get hub.

>> WADE WINGLER:  Cameron Cundiff is the creator of Alt Text Bot and is changing the face of how people who are blind or visually impaired interact in the world of Twitter.

Cameron, thank you so much for being on our show today.

>> CAMERON CUNDIFF:  Sure thing.  Thanks, Wade.

>> WADE WINGLER:  Do you have a question about assistive technology?  Do you have a suggestion for someone we should interview on Assistive Technology Update?  Call our listener line at (317) 721‑7124.  Shoot us a note on Twitter at INDATA Project or check us out on Facebook.

Looking for show notes from today’s show?  Head on over to eastersealstech.com.

Assistive Technology Update is a proud member of the Accessibility Channel.  Find other shows like this plus much more at www.accessibilitychannel.com.

That was your Assistive Technology Update.  I’m Wade Wingler with the INDATA Project and Easter Seals Crossroads in Indiana.