App Natural Augmented English allows English learners to experience spoken and written English in virtual encounters with natural scenes and dialogs. This easy-to-use app has both an intelligent camera and an intelligent microphone, which can be activated to view the written words for your surroundings and for the speech that you say and hear from the questions you ask. The words are presented one at a time and zoomed out in a large and eye catching format to attract your attention and create a more vivid learning experience. The size of the words, the rate of presentation, and the occurrence of speech with the written presentation can be precisely controlled by the learner.
The app requires an internet connection. It should be noted that all interactions with the app are completely anonymous, and the internet access is solely for the purpose of automated image and speech recognition.
The home screen shows four buttons, which are guide, settings, camera (see), and microphone (question). Touching the guide button gives the instructions for using the app. The settings icon allows you to adjust the size of the written words, the presentation rate, and whether speech should accompany the presentation of the written words.
The other two buttons on the home screen give pictures of a camera and a microphone. These two icons occur with the words see and question. The descriptions of these two functions are given in the see and the question sections of this guide.
Note that the app can only be used in landscape mode. When the picture of the camera is touched, the device is in camera mode. You explore your surroundings searching for a scene to photograph. There are two shutter buttons on bottom left and right of the screen. Either of these shutter buttons can be used to take a picture. Touching the USE button accesses the internet to find word descriptors for the picture. When a word is presented, its definition can be retrieved by touching the ??? button. Each printed word and its definition for the picture is presented on the screen one word at a time, which has been shown to optimize reading and learning. A new descriptor can be shown by touching the NEW button. Touching the bottom left pointing arrow button returns to the home screen. In the camera mode, there is also a camera icon on the bottom middle of the screen to allow a selfie view.
When the picture of the microphone is touched, the device shows a microphone and is in microphone mode. You can ask a question that will be automatically recognized. Once a question is prepared, touching one of the question buttons changes the screen to red, which means to start asking the question. The screen also shows two done buttons, which should be touched when you are done talking. The screen turns blue and presents the question. Then there is a pause and the answer is presented. Touching the bottom left pointing arrow button returns to the home screen.
There are many different types of questions you can ask. Here are a few examples.
You can retrieve results from Wikipedia.
Tell me about Ben Stiller.
You can convert various units of measurement.
How many miles in 7 kilometers?
You can ask for the meanings of words.
Ask for the spellings of words.
Ask for the meanings of ideas and concepts.
What is the meaning of global warming?
Ask about sports.
What is the Tour de France?
Ask about the date, time, and weather.
Is it raining in Washington, DC?
You can perform arithmetic operations.
What is 7 divided by 2?
There are many other questions that you can explore.
Sometimes the program will not recognize what you asked, or it will not have the information. In these cases, you will get the reply, sorry, I didn’t get that.”