Optical Design and Electro-Optics Integration in AR System Development

Working for many years on Augmented Reality (AR) optical system development, we have gained many insights about the development process of these systems. We have continuously modified and optimized our development process by implementing these insights. However, the most significant insight, in our opinion, raised many objections in the working environment and we felt we couldn’t fully implement it in the projects we were involved in.

We would like to share this insight, which we truly believe can be the key for the successful development of a sustainable AR system. Today, at JOYA Team, we are trying to implement it together with our customers and partners.

Traditional AR development process

Usually, the main focus of AR systems development process is on optical design. As a result, the key parameters for the project design guidelines are derived from imaging optics. Based on these guidelines and key parameters the optical designer proposes several optical concepts. Then, a trade-off between the concepts is done and the number of concepts is reduced to 1 or 2 primary concepts. Again, the trade-off between the concepts and the choices and decisions are made using the same key parameters. Usually, these key parameters are based on the imaging optical performances, such as: MTF, exit pupil size and FOV.

After the primary optical concept is formed, the project development team starts to look for an appropriate display source. In parallel, the optical designer moves on to detail design.

At this stage, the optical concept is already fixed and poses a constrain on the display source choice. In most cases, the display source is chosen according to the optical design requirements and consequently fails to meet other important requirements. These requirements, frequently considered as consequential, are categorized as non-imaging design requirements or system requirements: see-through contrast, color performances, display NA (Numerical Aperture), display CR (Contrast Ratio) and others.

At this point in the development process, when these parameters are evaluated and are most frequently not fully met, there are only few optional paths to continue the project development, none of them is optimal and they all require painful compromises:

1. Optical system re-design – Time, efforts and resources go down the drain. Starting from scratch is a hard decision, rarely feasible since so much effort and especially time is already invested up to this point.

2. Rely on technology improvements – For example, we can choose a low-performances display technology, hoping that this technology will be improved soon, and the new display product will match our design and provide the required performances. In this approach, the risks are very high, since we are dependent on technology improvement that has a lot of variables and uncertainties which are out of our hands. According to our experience, the technology improvement doesn’t meet the project expectations in the given time frame, or it is not replaceable with the current systems design. In many cases. there is still a need to redesign the optical system for a good match between imaging optics and the display. In this case, the loss of time, efforts and resources eventually is even higher, consequently the customer will not be satisfied either with the product or with the project cost & time frame.

3. Continue the current design development – As a result, the final product may comply to the imaging key requirements, but the non-imaging and overall systems performances would be unacceptable. The image quality may be satisfactory, but it will be hardly seen in the specific use-case scenarios or under specific background conditions. For example, the luminance may be too low or the colors washed-out or, in some cases, some unacceptable artifacts due to stray light may be seen. In this case, the customer or the final product user will be unhappy with the product.

Our approach for AR system development process

The need for a holistic development approach was sharpened in many development cases we were involved in. We believe that the optical development process of AR systems and smart glasses in particular shall be done in close and constant cooperation between the optical designer and the electro-optics designer (imaging and non-imaging aspects) from the start point through the whole development process.

The design process steps at our approach:

1. Define key parameters derived from the user’s needs

2. Form optical concepts that combine non-imaging and imaging aspects (with interfaces to other disciplines). Teamwork is crucial at this stage, and this is what makes the difference in finding optimal optical systems concepts that refer to all the key parameters.

3. Perform trade-off between the concepts according to the combined key parameters.

4. Continue the development of optical system with continuous feedback between imaging and non-imaging designers, including optical design and display unit design.

At first look, this process seems to take more time and requires more resources than the traditional process. However, in the long run, going through this process minimizes the projects risks. Usually, there is no redesign in this approach and the overall resources and development time is more efficient. Moreover, this approach produces the optimal working point for the system solution. This means, that the same performances are met with higher production yield, lower production cost, and wider optical manufacturers range.

This approach also provides the tools to the project management that can help them coordinate the development progress more efficiently and accurately:

· They can decide at an early stage, whether to continue system development using the current available display technology or wait for technology to be more mature.

· They can evaluate project risks more accurately and save time, money and resources.

· The expectations are well-coordinated, and the status is clearer for all the stakeholders: project management, designers, developers and the customer.

· Implementing this process, we have the best chance to produce an optimal working point that will give the best value for the customer.

We believe that this idea shall be considered in the design of any system involving optical and electro-optics elements.

The following diagrams show the traditional development process vs. JOYA Team holistic system approach.