Innovations and Patents - Mobile Device Voyager

A new generation of mobile multimedia terminals Mobile devices such as cellular phones and personal digital assistants are becoming more and more powerful, permitting an ever-broader range of functions and applications. Today’s communication terminals and wireless internet devices are limited, however, by their small visual displays. Office, internet and multimedia applications (sound, pictures and video) and gaming systems all use a well-sized quality screen. The problem is that a large screen also means a large device, high weight and high costs. What is needed – and is virtually assured of sound market success – is a device with the size of a cellular phone, the power of a personal computer and a virtual screen offering the dimensions and quality of a large desktop monitor. The product for which this patent application is being submitted is designed to resolve the current conflict between the small size of today’s mobile devices and the desire to view digital content on a full-format display. A virtual retinal display or VRD system, such as a small lens integrated into the mobile terminal, will scan the photons containing the image data directly onto the retina of the user’s eye. As illustrated in Figure 1, a photon source such as a laser diode or a light-emitting diode (LED) generates a beam of light (the photon source may actually be three such sources if a colour image is to be rendered). The beam of light is intensity-modulated by a modulation unit to match the intensity of the image being generated. A beam scanner then receives the modulated signal and scans it across the retina using a particular scan pattern. The terminal may consist of one VRD system or of two such systems for the projection of stereo images into the user’s eyes. Figure 1 The system can be fully integrated into a cellular phone either as a two-dimensional image system or as a stereo image system. An autonomous VRD system which can be docked onto an existing cellular phone like today’s camera module will also be available. The new system should represent a breakthrough for mobile terminals, offering: - small size with high resolution and high quality - low power consumption - a low price - stereo images and video in colour and - no irritation in sunlight. Research and development The kind of VRD system described above can be integrated into a cellular phone according to Professor M. Menozzi of the Swiss Federal Institute of Technology in Zurich, a specialist in the field. Its development by a team of specialists is expected to take about three years. I will be contributing to this project: - my ownership of the international patent application (patent pending) - my patent knowledge from the USA and Europe (as an inventor of US patents) - my extensive experience in mobile multimedia and communication devices and - my extensive experience in telecommunications technology.

The invention relates to an electronic device, in particular a mobile multimedia communication device, according to the generic term of claim 1. Such devices, e.g. mobile phones, PDAs (personal digital assistants), MP3 players, etc., are known in a wide variety of designs and are available on the market. These devices are equipped with a display for text and images, which consists of an LCD (liquid crystal display). Due to its dimensions, this only allows for moderate to poor image quality. It is therefore not possible, especially with small devices with small screen sizes, to view cinema films, videos or a photo album in acceptable quality on the displays. If larger displays are used, the device not only becomes heavier and more bulky, but also more expensive and requires more power. The present invention is based on the task of creating a device of the type mentioned above that guarantees excellent image quality even with smaller device sizes, is inexpensive to manufacture and has low power consumption, and can be used as a universal, mobile communication device.

According to the invention, this task is solved by an electronic device with the features of claim 1. Further preferred embodiments of the device according to the invention are the subject of the dependent claims. In that at least one VRD (Virtual Retinal Display) system is provided for processing an image or data source and projecting it in the form of laser beams onto the retina of the device user, and the device has a station designed for audio, video and data communication for receiving and transmitting signals via a radio network or other transmission devices, whereby the signals can be exchanged via the Bluetooth, Wireless LAN, GSM, GPRS, EDGE, UMTS, DAB system, the 4G or 5G system or via any telephone cable, radio or satellite network, a cost-effective device is created that can provide excellent image quality with high image resolution, in particular three-dimensional colour image resolution, that can be designed to be lightweight and easy to handle, and can therefore be used as a universal multimedia communication device anytime, anywhere. 

The invention is explained in more detail below with reference to the drawings. The following figures show purely schematic representations: Fig. 1 shows a principle of image recording and image transmission intended for a communication device according to the invention; Fig. 2 shows a block diagram of image recording; Fig. 3 shows a first embodiment of a communication device; Fig. 4 shows a second embodiment of a communication device; Fig. 5 shows a block diagram of image transmission; Fig. 6 shows a third embodiment of a communication device; Fig. 7 shows a block diagram of the communication device according to the invention; Fig. 8 shows a self-sufficient multimedia module connected to several peripheral devices; and Fig. 9 shows two multimedia modules that can be connected to a device. Fig. 1 schematically shows a principle used in the device according to the invention for transmitting three-dimensional images. Two electronic cameras (preferably CCD cameras) 11, 12 are used for image capture, as also indicated in Fig. 2, which capture a subject, e.g. the aircraft indicated in Fig. 2, from two different angles. The two partial images can be captured with a vertical or horizontal shift. This means that the two cameras 11, 12 can be arranged vertically or horizontally. The images captured by the cameras 11, 12 (stereoscopic half-images) are recorded and displayed in real time or at a later point in time via a VRD (virtual retinal display) system. If two VRD systems 15, 16 project the synchronised images captured by the two electronic cameras 11, 12 onto each of the viewer's eyes L and R, three-dimensional images (black-and-white or colour images) with high image resolution can be conveyed. With the aid of special recording techniques, three-dimensional image effects can also be achieved with a single camera. One camera is sufficient for two-dimensional image transmission. If only a two-dimensional image is required, two users can even view the same image information simultaneously if a laser beam is projected onto the eyes of both users via one VRD system each. With the aid of special technical means, however, it is also possible to convey three-dimensional image impressions with only one VRD system. More than two cameras could also be used to convey three-dimensional images. According to the invention, small, handy electronic communication devices can also be equipped with at least one VRD system 15 or 16 for processing an image or data source and projecting it onto the user's retina, thereby providing excellent image quality with high image resolution, in particular three-dimensional colour image resolution, which is completely independent of the size of the device. This enables the creation of a mobile, universally applicable communication device which is intended for audio, video and data communication between two or more users and/or between the user and external computer systems and/or peripheral devices. However, it can also be a so-called multimedia communication device that combines a video player, music player, PDA (personal digital assistant), laptop and game console in one device. Audio, video and data communication takes place via one or more communication modules integrated into the device. Data transmission preferably takes place via a radio network such as Bluetooth, wireless LAN, GSM (Global System for Mobile Communication), GPRS (General Packet Radio System), EDGE (Enhanced Data rates for GSM Evolution), UMTS (Universal Mobile Telecommunications System), DAB (Digital Audio Broadcasting), via so-called 4G (fourth generation) or 5G (fifth generation) or via any other radio system with a transmission speed of 20-100 Mb/s, or by other transmission devices – for example, via a telephone cable or satellite network. The device can be used worldwide and is capable of using multi-band (multiple frequency spectrums such as 900/1800/1900 MHz or other international frequencies). The device can be connected directly to a monitor, mouse, keyboard and printer at home or in the office, thus turning it into a work computer. Alternatively, the device can be converted into a portable laptop computer by connecting a keyboard. In the car, the device can serve as a navigation system. This means you no longer have to deal with a multitude of electronic devices – PC, laptop, PDA, mobile phone, video and music players, etc. – and constantly check where your data is stored. Everything merges into a personal companion that contains all your information and allows you to access the Internet and local networks anywhere and anytime. With a wide range of communication and multimedia functions, the device can also be used as an online game console in a particularly advantageous way. Two or more online game participants can also talk to each other via live video conferencing, for example, and form different game teams worldwide. However, two or more game participants could also be connected via the Internet, either in the same room or separated from each other geographically. An example of such a device 1 is shown schematically in Fig. 3. Device 1 has a housing 2 made of plastic, metal, a metal alloy or a sintered material so that it is lightweight, stable and inexpensive to manufacture and allows heat dissipation. EMC (electromagnetic compatibility) and ESD (electrostatic discharge) regulations are taken into account when selecting the material. Device 1 is equipped with the two cameras 11, 12 mentioned above and with the two VRD systems 15, 16, which enable three-dimensional image transmission. In this embodiment, cameras 11, 12 are arranged vertically one above the other, and VRD systems 15, 16 are arranged horizontally next to each other. The fields recorded by cameras 11, 12 are stored via an image multiplexer 13a (see block diagram in Fig. 2) on a video recorder or a video buffer 13b for further processing. The video signals can be forwarded to another device in real time via radio or cable network. Fig. 5 shows a block diagram of the dual VRD system present in device 1. Each VRD system 15, 16 has a photogenerator 17, 18 and a modulator 19, 20. The photogenerators 17, 18 generate photons that are modulated with video information in the modulators 19, 20. The respective modulator 19, 20 aligns the photons horizontally. The modulated photons are assembled vertically into a photon grid with the aid of a scanner 21, 22. The photons aligned to form a grid are projected directly onto the retina N of the device user via a projection optic 23, 24, which conveys the impression of an upright virtual image. For black-and-white images, one photon generator 17, 18 per eye is sufficient. For colour images, the photon generator 17, 18 houses three individual generators for red, green and yellow or blue light, which project RGY (red-green-yellow) or RGB (red-green-blue) modulated video signals directly onto the user's eye. In order to generate the most realistic, three-dimensional image possible, a VRD system 15, 16 is provided for each eye. Each VRD system 15, 16 synchronously transmits a field image to the retina N of the corresponding eye L, R, thus conveying a spatial image. A video drive system 13c demultiplexes the video signal and assigns the correct fields to the right and left VRD systems 15, 16. In addition, the video drive system 13c sends colour information to the photon generators 17, 18, horizontal and vertical synchronisation signals to the modulators 19, 20 and to the scanners 23, 24. In addition, a so-called eye-tracking system (i.e. a system for determining the position of the eye) 25, 26 can be provided for each eye L, R. For example, the exact position of the eye can be determined using an infrared beam. The position data is fed to the corresponding scanner 21, 22, which then makes the necessary correction so that the photon beam is always placed exactly on the pupil. The housing 2 of the device 1 accommodates an electronic circuit, storage media and an energy source in a manner not shown in Fig. 3 but mentioned below in connection with Fig. 7. The energy source can be formed, for example, by a battery, a solar cell generator, thermoelectric generators or a mini fuel cell and can be rechargeable. Solid-state memory (chip memory) is provided as working, data and programme memory. For capacity and cost reasons, device 1 can also be equipped with a hard disk. Biological and nanomechanical memory could also be used. Device 1 is also equipped with an antenna 27a (however, this could also be integrated invisibly into the housing 2). A so-called joystick 3 and buttons are provided as input devices. Fig. 4 shows another variant of a communication device 1' according to the invention, which essentially corresponds to device 1 according to Fig. 3 and also ensures three-dimensional image transmission. In this embodiment, the cameras 11, 12 are arranged horizontally next to each other and the VRD systems 15, 16 are arranged vertically one above the other. A keyboard 4, as in a mobile phone, is provided as an input device. Fig. 6 shows a third embodiment of a device 1’’ according to the invention, which is intended solely for two-dimensional image transmission and is therefore equipped only with a VRD system 15 and a camera 11. The device 1’’ has a sensor screen (touch screen) as an input device. 5 as an input device. However, a proximity switch that can be combined with a display could also be used as an input device. This switch could advantageously be inductive or capacitive, or switch by means of sound wave or infrared scanning, and could be activated by a magnet that can only be operated by the person who possesses the corresponding part and knows the corresponding functions. An acoustic signal can also be provided as an input device, whereby the device can be tuned to a specific frequency, frequency spectrum or voice. Brain waves could also serve as an input device, whereby one or more detectors are attached to defined locations on or around the head and the input command can be issued depending on the thoughts. In the office or at home, the device can be operated like a laptop or PC with a keyboard and mouse. All peripheral devices can be operated via cable or wirelessly using radio technologies such as Bluetooth, infrared, wireless LAN or Dect. Fig. 7 shows a block diagram of a possible device according to the invention, with a possible configuration of the overall system. Various modules are connected to a bus system 30. For stereo image capture, the two cameras 11, 12 mentioned above are connected to the bus system 30 via a control unit 31. For three-dimensional image display on the retina, the two VRD systems 15, 16 are connected to the bus system 30 via a control unit 32. The aforementioned electronic circuit (designated 35 in Fig. 7) advantageously comprises two separate processors 36, 37 (central processing units, CPUs) equipped with their own memory modules 38a (ROM), 38b (RAM) and 39a (ROM), 39b (RAM), respectively. The two processors 36, 37 can be operated simultaneously with different operating systems so that the advantages of a pocket operating system (such as Windows CE or Plam OS) and a PC operating system (such as Windows XP, Mac OS, Linux) can be utilised. Processor 36 with memory modules 38a, 38b and a real-time clock 33 (RTC) is intended for the PC operating system. Processor 37 with memory modules 39a, 39b and a real-time clock 34 is responsible for the pocket operating system. Both systems run independently of each other. A resource manager 40 allocates the system resources to the correct processor 36 or 37 depending on the priorities and user specifications. If, for example, a telephone call needs to be made or a stored address with the corresponding telephone number needs to be found, the user can start their pocket operating system (processor 37) and make the call immediately. but also start the PC operating system (processor 36) during the phone call so that they can save time and start working on the PC (e.g. word processing or spreadsheets) immediately after the phone call. The selective use of different operating systems and processors in a single device significantly increases user acceptance of so-called ‘all-in-one’ systems, reduces power consumption and thus extends operating time many times over. However, the device could also be equipped with only one processor and run on a single operating system, or contain more than two processors for more than two operating systems. The preferred rechargeable energy source is designated 41 in Fig. 7. It may consist of a battery, a charger, a solar cell generator, a mini fuel cell or a thermoelectric generator. The overall system may also have a hard disk, DVD, CD, MP3 or other music player (reference number 42 in Fig. 7) with a corresponding control unit 43. Furthermore, one or more SIM card or smart card readers 45 are operatively connected to the bus system 30 via a control unit 46. The SIM (subscriber identity module) card is a credit card-sized identification card for subscribers to a mobile phone service (GSM, GPRS, UMTS, etc.) and is generally referred to as a ‘chip card’ or ‘smart card’. It contains a chip with subscriber-relevant data and algorithms as well as the subscriber's access authorisation to the mobile phone network. Complex keys and algorithms for security systems can also be stored on the SIM card or chip. Due to the constant miniaturisation of devices, in most cases only the chip – removed from the SIM card – is installed in the device. The overall system also contains a receiving and transmitting station 27, which is formed by one or more radio modules for the wireless transmission of data and information and is connected to the bus system 30 via a control unit 47. The data can be transmitted, for example, over short distances via Bluetooth, wireless LAN and over long distances via GPRS, UMTS, DAB, 4G or 5G, or via satellite. An audio interface 49 for a microphone 50 and for loudspeakers 51 is coupled to the radio module and the bus system 30. The receiving and transmitting station 27 could also be formed by an infrared module for data transmission. For the reception of satellite data and for position determination, a position determination system 53 is connected to the bus system via a control unit 54. This positioning system 53 allows route navigation and location-based services to be used via terrestrial radio antennas or satellite systems. The device can also be equipped with a conventional LCD display 55 connected to the bus system 30 via a control unit 56. As already mentioned, a joystick 3 and/or a keyboard 4 or a touch screen 5, connected to the bus system 30 via a control unit 58, can be used for input. Fig. 8 shows a schematic diagram of a module 10 designed to transmit three-dimensional images, equipped with the two cameras 11, 12 and the two VRD systems 15, 16, which is designed as an independent unit and functions autonomously. The module 10 can exchange video signals and data with various peripheral devices, such as an external headphone set 60, a PDA (personal digital assistant), a cellular phone 62, a laptop 63 or other devices, via a radio or cable network. As indicated in Fig. 9, a module 10' designed to transmit three-dimensional images and equipped with the two cameras 11, 12 and the two VRD systems 15, 16 can be connected to or attached to a device 1a, e.g. a cellular phone or a PDA.

The use of the VRD method in the electronic devices according to the invention has the advantage that users with visual impairments (short-sightedness or long-sightedness) or with corrective lenses do not need glasses or other visual aids to read conventional screens or displays, as the projection takes place directly on the retina (the VRD method is self-correcting, so to speak). A further advantage is the relatively low cost and, compared to conventional displays, low Power consumption. An extremely low laser power (e.g. less than 100 nW) of coherent radiation is sufficient to produce a very bright image. This advantage is particularly striking in strong sunlight. While LCD displays are difficult or impossible to read even in normal sunlight, VRD systems produce razor-sharp images on the retina even in the strongest sunlight. The invention is sufficiently illustrated by the above embodiments. However, it could also be illustrated in other variants. The device according to the invention is particularly suitable for use in a vehicle, be it a car, truck, motorbike or similar, in which it can be used as a mobile device, either removable or fixed. When in use, the driver can concentrate on the road while the device simultaneously projects a virtually transparent image into their eyes.