12/09/2019

Using hands to “see” the world

hand

People with disabilities benefit from artificial intelligence developed by engineers from Los Andes.

Physical disabilities put those who are affected by them at a disadvantage. The purpose of accessibility is to create technologies that reduce this gap and make conditions more equal for them.

Around the world in recent years, different technological solutions have been created for the visually impaired, so they have more equal access to information. An example could be tactile images, which allow them to ‘read’ with their hands.

“They need to be configured to make sure they are well designed and can be printed. Compared to a normal image, they have very limited details. This is because touch and sight do not perceive with the same resolution. The former is not as sensitive to change as you touch point by point as if you are sweeping sideways, but with a single glance you can see the whole image”, says John Alexis Guerra, who has a PhD in Computer Science from the University of Maryland. He was a member of the team of professors from the Department of Systems Engineering and Computer Science.

John Alexis Guerra, professor from the Department of Systems Engineering and Computer Science
John Alexis Guerra, who has a PhD in Computer Science from the University of Maryland, is one of the team of professors from the Department of Systems Engineering and Computer Science.

During the past two-and-a-half years in which he has been a member of the Faculty of Engineering, Guerra has promoted his students’ interest in this topic and directed some undergraduate final research projects in this area. “Engineering is about using our projects to help people who need it, and, in general, the field doesn’t often work with people with disabilities”, observes Felipe Martínez—a systems and computer science engineer from Los Andes.

He was motivated by the teacher’s experience to carry out his undergraduate final research project entitled Tactile Graphics Finder. This is a platform constituted by a search engine that locates all the images found on the net related to the concept the user is searching for as well as a classifier that analyses them and establishes if they can be converted into tactile graphics. It also allows images to be uploaded and then to be identified.

“We believe that by using machine learning we can help to democratize tactile graphics, and, as such, help blind people to access information that is contained in digital images”, explains Martínez who developed the platform using Google’s AutoML, a tool to train learning models using transfer. This involves taking advantage of the pre-trained layers, which detect some basic characteristics in images, and retrain them so they can identify new categories. “To train the model, I created a database with good and bad images and, afterwards, we included it in our search engine”, he added.
We believe that by using machine learning we can help to democratize tactile graphics, and, as such, help blind people to access information
Felipe Martínez
Student

One step further

Carlos González later resumed Martínez’s work. “John suggested it and it was really interesting for me because it combined two topics that I really liked: accessibility and machine learning”, he highlighted.

The purpose of Tactiled, as the Venezuelan christened his undergraduate degree project, was to provide a virtual library of suitable images to convert them into tactile graphics. This platform looks for any image on the net that is requested by the user and then classifies it. It also allows photos and illustrations to be uploaded, but, in contrast to Martínez’s tool, if the options are not valid, you are offered a series of options that are.

Engineers from Los Andes University
Engineers from Los Andes found the ideal space in classrooms to apply their knowledge and use artificial intelligence to help other people. From left to right: Ricardo González, Carlos González, Antonio de la Vega, and Felipe Martínez.

Another advantage is that it works collaboratively: it is nourished by the images that specialized users on the topic have signaled as being appropriate for being tactile images.

“We gave them a series of images found on the internet and they decided if they were good or not. When we started the project, the model’s reliability was 88%, but, thanks to this retraining process, it increased to 94%”, pointed out González. “The idea is that this platform turns into a reference archive where anyone can find images that they can use as tactile images”, adds Venezuelan student and project assistant Ricardo González, who is studying Systems Engineering and Computer Science.

A collaborative spirit

Another example of a technological advance that improves accessibility for the visually impaired is screen readers: programs that describe elements that appear on computer or cellphone screens to the user while they browse the web.

However, they are often not very efficient as the majority of webpages are not created with the needs of this public in mind; therefore, there are many accessibility flaws.

One of the most common errors is that the images are not well tagged, which prevents the screen reader from describing it correctly. Another is the absence of color contrast in different elements of the page, which makes it difficult to read for anyone who has vision problems.

To find a solution to these problems, Antonio de la Vega created Magically, which is a platform that helps to create accessible webpages. These pages respect the established standards so a screen reader or another accessibility technology can efficiently read it.

Magically analyses the webpage and checks if it has accessibility errors. If this is the case, it shows an interface with a list of the problems that need to be resolved; the user can fix them even with no knowledge of programing. This new configuration—with the corrections—is recorded as a browser add-on, and when someone else enters the platform and looks at the same page, the tool automatically provides the solution provided by the previous user. “It has this crowdsourcing component, which means the users themselves create the solutions to accessibility problems”, explains Antonio.
The tool has two components: The first is the extension that is installed in the browser, which determines if another user has already made corrections to the page, translates them into code, and then applies them. The second is the web application, which shows the list of problems to solve. The two components communicate with each other to transparently correct accessibility problems while the person is browsing.

“Between 10% and 15% of the global population has some form of disability. From a software developer’s point of view, all possible platform users must be taken into consideration. However, the social element also needs to be contemplated as everyone should have access to information”, emphasizes de la Vega. “These academic spaces are ideal to carry out projects like this as, if students don’t do it, no one is going to do it. They have the tools to solve these problems, they have this superpower, which comes with a responsibility that cannot be ignored. What can they do to change the world? to look further than what is in front of them, instead of just thinking about their own personal gain”, concludes Guerra.

Tactiled: artificial intelligence to democratize information


(1) Use a pre-existing model to classify images; (2) find good and bad images to convert them into tactile graphics; (3) insert new categories into the machine leaning model and retrieve it; (4) upload the new model into the browser; (5) the platform shows a series of tactile images or any image the user is searching for; and (6) transform the bad images into acceptable images.

Iris and BTactile.com


Throughout his career, John Alexis Guerra has always had an interest in accessibility.

For their final undergraduate project in systems engineering in 2003, a group of engineers from the Universidad Tecnológica de Pereira—including his wife María Fernanda Zúñiga who was also a professor in the department—created Iris. This is an external device that allows blind children to see images and colors using their hands.

Iris is currently used in three educational institutions in Pereira, Santa Rosa de Cabal, and Bogotá.

Additionally, together with Zúñiga, they have been developing BTactile.com for the past five years. This is an image search engine that allows tactile images to be found that come from eight different countries’ archives, which are created by experts in each country. “It is a search engine with the greatest number of tactile images in the world—it has around five-thousand images—and is well-known in the tactile image and blind community throughout the world”, indicated Guerra.

Interesting Facts


What is a tactile image? A tactile image needs to have exactly the same amount of information as the original so its meaning is preserved; however, unnecessary data that hinder the individual being able to understand them should be avoided. Some important characteristics are: they should have simple lines without many details, they should avoid the use of perspective and should avoid many intersections between the lines.

How are tactile images made?

  • Braille printing: this involves making a map of the pixels in a digital image and converting them into braille points, which creates a relief image of the original.
  • Swell paper: is a type of paper coated with microscopic polystyrene capsules that, when exposed to heat, makes the painted areas grow. This creates a relief image.
  • Plastic embossing film or German film: this is a plastic paper that, when embossed, becomes raised and creates a tactile diagram.
  • 3D printers: using a 3D model, the printer creates the figure.


 

Related news

MADI logo with techno brain shape
28/07/2020

Registration will open from February 2021, and the first cohort will start classes in August 2021.

Ver más
 Image of the metabolomics laboratory
23/10/2019

The country’s first metabolomics center, which is one of the first in Latin America, was inaugurated in the Universidad de los Andes.

Ver más
Asbestos roof
24/07/2019

A study led by Los Andes discovered a high rate of cancer together with the presence of asbestos in the subsoil in the municipality of Sibaté, Cundina

Ver más

Otras noticias

Share

  • Logo Facebook
  • Logo Twitter
  • Logo Linkedin