The Flemish Sign Language bilingual dictionary - part 2

As mentioned in part 1 of this project, the AE team helping out the Flemish Sign Language Centre (FSLC) took a short break from their project to attend the AE Hackathon.

Missed our first part about the FSLC’s explanatory dictionary? No worries, you can still read it here!

Want to follow our story in Dutch Sign Language? Take a look at our video!

The timing of the break was no coincidence. After all, this year’s annual innovation marathon saw another AE team try to teach an AI program how to recognise sign language gestures and translate them into regular Dutch – which was precisely one of the ideas which had sprung to mind in the initial phases of the explanatory dictionary project. The Hackathon, then, was the ideal opportunity to take a closer look at this particular technology. The result was impressive, to say the least. However, the Hackathon did show that AI technology is currently still too challenging to include in the dictionary, as there is not enough training material available. Consequently, AE and the FSLC decided not to spend any more money on it for the time being.

Want to know what happened during the Hackathon? Read all about the sign language app in our Hackathon ebook! 

User feedback

Following the short interlude, the team continued to work on the visibility trajectory, striving to create a user-friendly interface for an application that would be perfectly adapted to the FSLC’s target group of deaf and visually impaired people. An important change to the original plan, however, was that the production environment would become open to users (something the Sign Language Centre facilitated themselves). This enabled the team to collect feedback from the target audience. Users were invited to share their thoughts via email or video, or simply by giving the FSLC a call. All feedback was evaluated on a weekly basis and priorities were set accordingly.

In a penultimate sprint, the AE team added a feedback feature to the application – a wake-up call for the Flemish Sign Language Centre, as it turned out. Philippe De Croock, Product Owner for this project, explains: “When we wanted to include a text field for users to leave feedback, the FSLC suddenly realised the GDPR had yet to be taken into account. The legal department was set to work, resulting in an extensive Privacy Policy and Cookie banners.” A seemingly minor intervention with far-reaching consequences for the centre.

The changing search algorithm

Philippe closely monitored the changes: “We found ourselves confronted with the fact that people’s search behaviour is in fact not as easy to pin down as you’d expect. For example, there’s a difference between filtering and searching. People are used to entering a few keywords in Google and then being presented with a series of relevant results. Our system, however, initially worked with tags and multiple search terms that could have an OR relationship. That makes things a lot more complex, as our user tests clearly pointed out.”

The team therefore opted for a complete overhaul of their previous search algorithm. Philippe explains why: “In the end, we decided to use search terms to specify the results instead of broadening them, by using an AND relationship. Kind of like Google does. It’s a completely different way of presenting things.”

Fortunately, their hard work was not lost. Philippe: “We also added a functionality in case there were no results to be displayed, making sure suggestions would then still show by means of an OR relationship. This is a great example of how sometimes you end up developing things that don’t fully match, but that doesn’t necessarily mean it’s all been for nothing. On the one hand, there’s the research value of it all, and on the other, you’d be amazed at how often you can reuse an idea by taking a creative approach.”

A user-friendly dictionary

Philippe is proud of what the team accomplished: “We now have a dictionary that is based on the 1.0 version yet comes with many user-friendly features. People can not only find translations for Dutch search terms, but can also base their search on icons or categories. We also made it possible to filter searches by provinces, since gestures are often region-specific, similar to dialects. This function is especially useful for interpreters.”

In addition, from now on you can start your search from the characteristics of a certain gesture, both in terms of hand shape and location on the body. This was previously only possible via a so-called ‘ASCI code’. Finally, the team collected extensive information for each gesture. At a glance, you can now view typical characteristics and the category a gesture belongs to, as well as the variations that apply for each province. Students can also quickly find out what similar gestures exist, so that they can easily distinguish between gestures that may look alike but have completely different meanings. Finally, compound words (e.g. “football square”) are now displayed as well.

The newly updated dictionary promises to have a major impact, both on the deaf population and on the FSLC itself. The project will definitely bring the centre closer to its audience, says Lisa from the FSLC:

“This project has provided us with valuable insights into our target audience. We are very grateful for our smooth and pleasant collaboration with AE. Because Philippe always communicated so candidly with us, we saw all our expectations met. We are very satisfied with the final result and can’t wait to see how the dictionary will be received by our community.”