Abstract- Individuals with disabilities experience different forms of exclusion and have a hard time part taking in everyday activities. The main aim of this paper is to help improve communication skills through assistive technology application for individuals with disabilities that make it difficult for them to understand and learn. It aims to achieve this result through machine learning and deep learning tools and techniques. This application has functionalities such as sentence correction, speech to text conversion and text to speech conversion. Since all of these functionalities and features are available in one single application, it can be used by disabled individuals in their day to day activities to help communicate and understand others with ease.Keywords- Machine Learning, Sentence Correction, Sequence to Sequence Learning, Speech to Text, TensorFlow, Text to SpeechI. INTRODUCTIONIndividuals with disabilities experience different forms of exclusion.
This may cut them off from education, health, and social services, as well as limit their participation in family, the community and the society. This may have a serious impact on employment opportunities and participation in the society as a whole.. Supportive services and technology can help individuals with disabilities to take their place in society and contribute to their family, the society and the community. The main goal of this paper is to use ML techniques to create an Assistive Technology for Specially Abled Individuals.
Our proposed application aims to give the differently or specially abled a shot at having a normal, self-made life. The purpose of this project mainly revolves around people (mainly adults), who have poor knowledge of the English knowledge or have disabilities such as dyslexia, autism, speech and hearing impairment.A. Barriers to Specially Abled IndividualsSpecially abled individuals face various disparities and unsettling challenges to the availability of academic, social, and community participation in lower and middle income countries. They are put through additional discrimination and social exclusion based on gender, language, age, religion, social status, ethnicity, living in conflict zones and other factors.
Women or girls with disabilities are particularly at risk of discrimination and abuse. Children with disabilities have lower rates of primary school completion than those without and in many cases assistive technology can help them to further develop their learning capacity. Specially abled individuals are more likely to be unemployed and to live in poverty in adulthood. Households with a member who is specially abled have a higher risk of living below the poverty line. Without Assistive Technology for specially abled individuals, there is a need for accessibility and support which leads to greater dependency on family members or those that are closely related to the specially abled individual. In this case, family members may lose additional income because they become primary caregivers.
In some cases, siblings have to play the role of primary or secondary caregivers, stripping them of the opportunity to go to school and participate in the community.B. Assistive TechnologyAssistive technology is used as an umbrella term for both assistive products and related services. There are various definitions of assistive technology.
They are as follows:1. The International Classification of Functioning, Disability and Health (ICF) defines assistive products and technology as any product, instrument, equipment or technology adapted or specially designed for improving the functioning of a person with a disability. 2. Obtained from the The International Classification of Functioning, Disability and Health, the International Organization for Standardization (ISO) defines assistive products more broadly as any product, especially produced or generally available, that is used by or for persons with disability: for participation; to protect, support, train, measure or substitute for body functions/structures and activities; or to prevent impairments, activity limitations or participation restrictions. This includes devices, equipment, instruments and software.
Services related to assistive products include referral, assessment of the individual, prescription, funding, ordering, product preparation, fitting/ adjusting of the product to the individual, training of the individual or family members, follow-up, and maintenance and repairs. To diagnose and monitor underlying conditions that negatively affect functioning, medical devices and clinical expertise may also be required. Each and every type of assistive technology has its own individual assessment methods. For example, the way in which the product is adapted, modified or fitted. The personnel involved in the service delivery must have the necessary knowledge to prevent potential harm caused by incorrect assessment and fitting. This is very important because in the same way that appropriate services can have a substantial impact on the outcomes of using assistive technology, delivery personnel with no relevant knowledge of the assistive technology or its diagnosis may give a complete incorrect assessment which can have very negative impacts. C.
Benefits of Assistive TechnologyWhen appropriate to the user and the user’s environment, assistive technology is a powerful tool to increase independence and improve participation. It helps individuals communicate more effectively, become mobile, see and hear better, and participate more extensively in learning activities. Furthermore, assistive technology reinforces individuals to gain access to and enjoy their rights and do things they think are significant. Hence, bridging the disparities between individuals with and without disabilities.
It provides a mechanism for accessing to and participating in social, recreational and educational opportunities. It not only empowers greater physical and mental function and improves self-esteem but it also reduces costs for educational services and individual supports. Benefits in areas such as education, mobility and health have been connected to the use of assistive technology. By improving accessibility to education and increasing achievement in schools, assistive technology can have a positive socio-economic result on the lives of specially abled individuals.
II. MOTIVATIONThe global need for assistive technology for individuals has not yet been adequately quantified. Studies show that around 0.5% of the population need prosthetic or orthotic devices, about 1% need a wheelchair, and about 3% need a hearing aid but do not have access to them. These requirements differ between countries and between regions within a country because of factors such as discrepancies in age distribution and ubiquity of various other impairments. WHO estimates that only 5-15% of assistive technology needs are met in many developing countries.
Studies done in Malawi and Namibia show that more than 80% of those who need assistive technology do not have access to it. In situations of crisis and emergency, individuals with disabilities suffer from a triple disadvantage: they experience the same impact as others, they are less able to cope with deterioration of the environment, and responses to their needs are postponed or disregarded. To reduce the effect of emergencies or a crisis, specially abled individuals may need assistive technology to be alerted or to escape any sort of danger before it occurs. It can also simply be used to be able to carry out activities of daily living essential for their health and survival.
It is also seen that there is a very high rate of unemployment in disabled individuals. Below is the unemployment rate in disabled and non-disabled individuals.Fig 1.
Unemployment rate differences between disabled and non-disabled individualsAs you can see from the above, disabled individuals are presently, at a massive disadvantage.In this paper, we wish to provide an application that can be used by individuals with any sort of learning disabilities to blur the line between disable individuals with the normal working class. The main aim of this paper is to help improve communication skills through assistive technology such as text to speech, speech to text, and sentence correction.III.
RESEARCH BACKGROUNDThere are several other authors that have worked in the field of machine learning tools and techniques, the social impact of such an application and how such applications have a positive impact on disabled individuals, their family and the society as a whole. Mgr. BcA. Tereza Parilová et al.
(2015) proposed “Automatization and Personalization of Assistive Technology Processes for Users with Dyslexia.” They discovered that apart from Braille, the Text to Speech technologies (TTS) are the only possibility to make all written information available to the visually impaired and blind. They also found that the elimination of reading limits the psychological development of an individual and can result in worsened socialization and inuence on the quality of life of a person with dyslexia. It was concluded that TTS will have positive effects on individuals with dyslexia.
Various authors from Kurzweil Education Systems proposed “Using Technology as a Solution for English Language Learners in Higher Education” and discusses the problem that man post secondary schools have where they fail to provide the individualized support English Language Learners need to meet multiple course requirements. Most institutions readily acknowledge the need for additional support, but often lack the necessary staff or resources to provide it. The identified that the solution to this is text-to-speech technologies which offer a cost-effective and efficient way to provide English Language. It enables students to simultaneously read and listen to any kind of digital or scanned printed material as well as provide a host of on-line reference and study skills tools to strengthen reading, writing and comprehension.Paul Barham et al. (2016) proposed “TensorFlow: A System for Large-Scale Machine Learning” and explained that TensorFlow is a machine learning system that operates at large scale and in heterogeneous environments.
The paper describes the TensorFlow dataflow model and demonstrates the performance that TensorFlow achieves for several real-world applications. It has high performance and customizability, but it comes at the cost of a much steeper learning curve and solid understanding of machine learning and mathematical concepts especially linear algebra and calculus.Shaona Ghosh et al. (2017) proposed “Neural Networks for Text Correction and Completion in Keyboard Decoding.” This paper proposes a sequence-to-sequence neural attention network system for automatic text correction and completion.
Given an erroneous sequence, the proposed model encodes character level hidden representations and then decodes the revised sequence thus enabling auto-correction and completion. Further, what makes the problem different from the generic language modelling is the simultaneous text correction and completion. This is achieved by a combination of character level CNN and gated recurrent unit (GRU) encoder along with and a word level gated recurrent unit (GRU) attention decoder. IV. IMPLEMENTATIONThe application proposed in this project provides a platform for individuals with disabilities to communicate, learn and understand through its multiple functionalities. With this application, users can input a text document, any and all spelling mistakes in this document will be corrected and outputted back to the user.
The user can then hear the corrected text document through the applications text to speech feature. There is an additional speech to text functionality. All of these functionalities help disabled individuals perform their daily activities with ease.A.
Sentence CorrectionOne of the main features of this application is spell checking. A document can be uploaded and the application will not only check for spelling mistakes, but correct these mistakes and output the corrected document. This can be a great tool for any kind of disabled individuals and even for students. Because of its high performance and capabilities, we used TensorFlow for this feature. TensorFlow is a machine learning system that operates at large scale and in heterogeneous environments. It uses dataflow graphs to represent computation, shared state, and the operations that mutate that state.
It maps the nodes of a dataflow graph across many machines in a cluster, and within a machine across multiple computational devices. TensorFlow enables developers to experiment with novel optimizations and training algorithms. For this application, we explored TensorFlow’s seq2seq library. An encoder that processes the input and a decoder that generates the output are the two recurrrent neural networks (RNNs) that the seq2seq model consists of. This basic architecture of TensorFlow’s seq2seq module is depicted below.Fig 2. Basic architecture of TensorFlow’s seq2seq moduleEach box in the picture above represents a cell of the RNN.
The encoder and decoder can share weights but it is common that the encoder and decoder use a different set of parameters. Sequence-to-sequence models have successfully used multi-layer cells as well. In the figure of the basic architecture shown above, every input has to be encoded into a fixed-size state vector, since that is the only thing passed to the decoder. To allow the decoder more direct access to the input, an attention mechanism was introduced. This allows the decoder to peek into the input at every decoding step.In our code, the data is first cleaned and organized into sentences before it is fed into the model. To track the performance of this model, data will be split into a training and testing set.
The testing set will be composed of 15% of the data. There is a function that will convert the sentences to sentences with mistakes, which will be used as the input data. Mistakes are created within this function in one of three ways:the order of two characters will be swapped (hlelo ~hello)an extra letter will be added (heljlo ~ hello)a character will not be typed (helo ~hello)The likelihood of either of the three errors occurring is equal, and the likelihood of any error occurring is 5%. Therefore, one in every 20 characters, on average, will include a mistake.
Typically one would create their input data before training their model, which would mean they have a fixed amount of training data. However, in this application, new input data is created as we train our model. This means that for every epoch the target (correct) sentence a new input sentence should be received. Using this method, we have an infinite amount of training data. The more training data, the more accurate is the final outputted document in terms of spelling.
B. Process of Sentence CorrectionThe project is divided into the following sectionsLoading the DataPreparing the DataBuilding the ModelTraining the ModelLoading the data simply refers to the act of collecting varied sentences and words so that the final data model obtained is more accurate and can handle huge varieties of words and sentences in the English language.Preparing the data model is the process wherein the data is cleaned of any noise such as punctuations and unknown characters.
This is to ensure that the training data is accurate and not faulty.Building the model is the process of building a graph, setting the set of rules and laying the groundwork so that the model will flow for the best possible training and prediction.Training the model is most accurate and fast when performed on the GPU simply because it has a higher processing capacity and can handle large amounts of data at a time when used properly.C. Text to Speech conversionThis is another feature that the assisted technology application will have. Readily available Python modules are used to code implement this feature. Pytsx is a cross-platform text-to-speech wrapper.
It uses different speech engines based on your operating system:nsss – NSSpeechSynthesizer on Mac OS X 10.5 and highersapi5 – SAPI5 on Windows XP, Windows Vista, and (untested) Windows 7espeak – eSpeak on any distro / platform that can host the shared library (e.g., Ubuntu / Fedora Linux)In this application, this feature can be used in 2 ways.1) User can directly give text to be converted to speech.2) The text that was corrected from the first feature can be converted in to speech.D.
Speech to Text ConversionThe third feature in this application is Speech to Text conversion. In this case also, there are Python modules that can be used for its implementation. Google has a Speech Recognition API that converts spoken text (microphone) into written text (Python strings).Simply speak in a microphone and Google API will translate this into written text. The API has excellent results for English language so it will be very easy as well as accurate to use. This feature will be useful with individuals with hearing disabilities, writing disabilities, for note making, listening to a lecture, etc.
V. CONCLUSIONThis paper highlights the issues faced by disabled individuals as well as discusses the positive impact that assistive technology can have on these individuals, their families and the society as a whole. Presently, there is a massive line separating disabled individuals from non-disabled individuals in every industry and in all spheres of life. The assistive technology proposed in this paper can help better and improve communication skills though sentence correction, text to speech and speech to text features. This is achieved by Machine Learning techniques such as TensowFlow’s seq2seq library and other Python modules. Ultimately, this application helps disabled individuals work as non-disabled individuals would by providing a platform with all three important features in one application. VI. FUTURE SCOPEThe project has a large future scope, it can be expanded in multiple ways.
One direction involves including various languages and dialects focusing on region wise english communication skills. Another would be implementing a grammar check in sentence correction as well as the spelling check that already exists in the application. The project can also be expanded to include other features such as helping understand and solve basic mathematics problems etc.VII. REFERENCESUNICEF. The state of the world’s children 2013. Children with disabilities.
New York: United Nations Children’s Fund; 2013.May-Teerink T. A survey of rehabilitative services and people coping with physical disabilities in Uganda, East Africa. Int J Rehabil Res. 1999;22(4):311-6.WHO.
World report on disability. Geneva: World Health Organization; 2011.Convention on the Rights of Persons with Disabilities. New York, United Nations, 2006.WHO. International classification of functioning, disability and health (ICF).
Geneva, Switzerland: World Health Organization; 2002.Andrich, R., ; Caracciolo, A.
, (2007) Analysing the cost of individual assistive technology programmes. Disability ; Rehabilitation: Assistive Technology, 2(4), 207-234.Bensi, N., Bitelli, C., ; Hoogerwerf, E-J., (2011) Assistive technologies and other solutions for independence: cost or investment?.
Proceedings of the 11th European Conference of the Association for the Advancement of Assistive Technology in Europe, Maastricht, The Netherlands, August/September 2011: p. 270 – 277.Digital Strategy for Schools (2015-2020) Enhancing Teaching, Learning and Assessment Retrieved on September 21st 2016. https://www.education.ie/en/Publications/Policy-Reports/ 34 Digital-Strategy-for-Schools-2015-2020.pdfWorld Health Organisation ; World Bank (2011) World Report on Disability.
Geneva,CH;WHO.American Library Association (2001), “Library Services for People with Disabilities Policy”, viewed 21 October 2014.Bekiares, S.E. (1984), “Technology for the handicapped: Selection and evaluation of aids and devices for the visually impaired”, Library Hi Tech, Vol.2, no.1, pp.57-61.
Berkeley, D., Kressin, L. and Oberlander, C. (2007), “Deploying assistive technology across campus: A collaborative approach”, Proceedings of the 35th Annual ACM SIGUCCS Fall Conference, New York, USA, 7th-10th October, pp.11-15, viewed 23 September 2014.Ethridge, J. (2005), “Removing barriers for visually impaired users through assistive technology solutions”, Mississippi Libraries, Vol.69, no.
4, pp.82-85, viewed 10 October 2010.Ministry of Social Justice and Empowerment (2014). “National Policy For Persons with Disabilities”, Ministry of Social Justice and Empowerment, viewed 05 November 2014.Jeffrey Dean, Greg S. Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V.
Le, Mark Z. Mao, Marc’Aurelio Ranzato, Andrew Senior, and Paul Tucker. Large scale distributed deep networks. In International Conference on Neural Information Processing Systems, pages 1223–1231, 2012.Ilya Sutskever, Oriol Vinyals, and Quoc V Le.
Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112, 2014.Minsoo Rhu, Natalia Gimelshein, Jason Clemons, Arslan Zulfiqar, and Stephen W Keckler. Virtualizing deep neural networks for memory-efficient neural network design.
arXiv e-prints abs/1602.08124, 2016.Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio.
2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 . Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting of the International Journal of Artificial Intelligence and Applications (IJAIA), Vol.
8, No.5, September 2017Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks.In Advances in neural information processing systems. pages 3104–3112. Ilya Sutskever, Oriol Vinyals, and Quoc V Le.
2014. Sequence to sequence learning with neural networks.In Advances in neural information processing systems. pages 3104–3112. Alexander J Ratner, Christopher M De Sa, Sen Wu, Daniel Selsam, and Christopher Ré. 2016.
Data programming creating large training sets, quickly. In Advances in Neural Information Processing Systems. pages 3567–3575.
Minh-Thang Luong and Christopher D Manning. 2015. Stanford neural machine translation systems for spoken language domains. In Proceedings of the International Workshop on Spoken Language Translation.