MatrixScan Find is an API that enables quick item location using AR overlays


MatrixScan, a Scandit SDK solution, enables smart devices to capture multiple data points simultaneously using an augmented reality (AR) interface. One of its APIs, MatrixScan Find, is designed specifically for AR-based item search. It scans multiple items at once and uses an AR overlay to highlight the matching item, offering a clear visual cue. This approach significantly reduces search time and improves accuracy compared to traditional item-by-item searching.

Employees in retail, logistics, and fulfillment centers often face challenges locating items—especially similar-looking ones such as spices, cosmetics, shoe boxes, or generic brown packages. Time constraints, combined with limited information (e.g., acceptable substitutions), can lead to inefficiencies, errors in fulfilling online orders, or inadequate service for in-store shoppers. Recognizing this challenge, we identified an opportunity to leverage camera vision technology to alleviate the time constraints of item location.

 
 
 
 
 
 
 
 

Who are our target users?

MatrixScan Find is designed specifically for frontline workers who spend considerable time locating items. These workers often search for items by reading labels or scanning barcodes from large stacks on shelves or in backrooms—an experience akin to finding a needle in a haystack. This challenge is especially pronounced with similar-looking items, leading to frequent struggles and human errors in picking or packing tasks. As a result, item search and sorting tasks often remain unfinished within a shift or delays in searching contribute to shipping delays.

With the growing adoption of ‘bring-your-own-smartphone’ policies, where employees use smartphones instead of dedicated scanning devices, a significant opportunity emerged to enhance the item search process using augmented reality (AR) technology.

 
 
 
 
 

What are the ideal contexts for
AR Item search?

MatrixScan SDK is designed to support a wide variety of contexts such as order picking & fulfilment, van loading & destaging, inventory & shelf management, transportation, and so on. To make the AR search experience ideal and efficient, I reviewed customers’ physical setting photos and recommended focusing on a couple of use cases that meet following criteria:

  • End users knows which item they need to find and where it’s located in ballpark, otherwise they would spend too much time holding their phone.

  • Barcodes on items are facing one same directions, or the search experience would be challenging.

After the context analysis the team decided to target use cases that meet all requirements:

  • In-store pickup (users have online order lists and pick up items from shelf)

  • Van loading (gig delivery drivers load packages to their truck from depot)

  • Item sorting in warehouse/backroom

 
 
 
 
 

xfn Team design sprint

I led a design sprint with a cross-functional team to map critical user journeys across three target contexts and illustrate how item search integrates seamlessly into users' workflows. Our analysis revealed two primary bottlenecks: (1) when the desired item is out of stock or misplaced, and (2) when locating an item on densely packed shelves. To address these challenges, we explored enhancing the existing scanning capabilities by incorporating an AR search experience. This integration allows frontline workers to perform "search-to-scan" tasks efficiently within Scandit's ecosystem.

 
 

Based on the journey mapping results, I designed the first concept of MatrixScan Find out-of-the-box UI. I made a few hypotheses on this prototype and wanted to test with end users:

  • Users may want the ability to pause the screen immediately upon locating a target item.

  • Regarding the method of pausing, users might prefer a manual pause option, or the app could automatically pause the screen when an item is found.

  • Additionally, users would require visibility into the items they need to search for and the ability to monitor real-time progress throughout their workflow.

 
 

Coping with Limitations in USER TESTING


Testing the concept prototype with our ‘actual’ target users often presents challenges for B2B solution companies because in general they are not allowed to reach their end users. I had to found a leeway to find alternative participants who have a good understanding of item search in retail & delivery contexts. Instead of using a conventional user research methodologies, I ‘hired’ between 8 to 12 gig workers through Coople, a part-time hiring platform, to let them use my concept prototype in our lab and gather feedback on it.
 

 
 
 

The testing result was mainly positive - participants easily found target items in all contexts. Especially, they were appreciated for being able to pause the live view, as it’s more ergonomic for double-checking items to search for. As anticipated, they prefer manual pause to automatic pause, because auto-pause is disorienting if found item is covered up. 

While I could validate all the hypotheses, I also got other valuable feedback through the testing:

  • Users didn’t feel that the item list is necessary while searching for items, but they did want to see search results at the end of search process.

  • In the frozen view, users wanted to match the list of items with blue highlights to connect two different information.

One another important takeaway from the user testing is that workers often need to search for items during their workflows, so I propose enhancing the regular scanning functionality of Scandit by integrating an item search experience. This enhancement will allow users to locate items within context, ensuring a seamless experience within the Scandit ecosystem. Frontline workers start their work flow from normal barcode scanning, and in case they need to search for an item, they enter MatrixScan Find.

In the 2nd round of design iteration, I embedded an entry point to MatrixScan Find on SparkScan, a scanning widget solution that can be loaded on any app surfaces.

 
 
 
 

ITEM SEARCH AS A PART OF WORKFLOW

The cross-functional team conducted an internal review of the R2 design and provided feedback on several points. Firstly, the UI for the entry point of MatrixScan Find is not intuitive and lacks discoverability due to its icon. This challenge arises primarily because users may not be familiar with this functionality from various applications or use cases. Also with the current design we can only provide the item search functionality only through SparkScan.


So in the final round, I introduced a new filter carousel UI and moved the entry point at the bottom of the screen. With the carousel UI, it’s easier to expand our scanning capability in the future and jumping between the capability would be more seamless.

 
 
 
 
 

Final design - Expanding ecosystem

MatrixScan Find capability became a part of the Scandit ecosystem in the final version. With the new UX framework Scandit Data Capture SDK can help users find items in both single and multiple barcode scanning contexts. It also supports from item search to item scan/receive/pickup in a single step.

 
 
 
 
 
 

Sharing New UX Research Methodologies at Interaction 23


After the project, I was invited to Interaction 23 and gave a talk about ‘leeway’ UX research methods that can still help designers gain valid insights about their target users when they are not be able to reach them. Surprisingly, I received a lot of questions from the audience facing similar access challenges. This highlighted that limited access to end users is a more prevalent design challenge than I had anticipated. Nevertheless, I learned from my own experiences that designers and researchers may find alternative methods to gain insights into their (potential) target users.