Are you running a Shopify store and find yourself wondering, "I want to change themes, but what if my sales drop?" or "I want to improve my design, but I'm not sure if it will actually be effective?"
To solve these challenges, Shopify's official AI simulation app, "SimGym," finally became generally available in Japan in March 2026!
With SimGym, AI acts as a virtual shopper, navigating your store and allowing you to verify the impact of changes before they go live.
This article provides an overview of SimGym, its plans, and a report on our experience using SimGym for theme analysis and comparison tests. If you're looking for tips to improve your store, please read on to the end.
What is SimGym?
SimGym is an AI simulation app provided by Shopify. It was announced at Shopify Winter '26 Edition (also known as RenAIssance) and has garnered attention as a groundbreaking tool for simulating e-commerce site changes before they go live.
How SimGym Works
At the core of SimGym are AI agents called "AI Shoppers." They are trained on billions of transaction data points across the entire Shopify network and navigate stores just like real customers.
The basic flow of SimGym consists of the following three steps:
- Select Themes to Compare — Variations of a theme are compared to your current theme.
- Start Simulation — AI shoppers are assigned, complete tasks, and share feedback on themes.
- Review Results — Results are summarized with insights to help inform your next steps.
What Can SimGym Do?
- Theme Analysis: Have AI shoppers browse live or unpublished themes and get feedback on various areas such as the homepage, product pages, collections, and cart.
- Theme Comparison (A/B Testing): Compare two themes side-by-side using AI to verify which has a higher conversion rate.
- Receive Improvement Suggestions: AI shoppers specifically point out UX issues in your store and provide tips for improvement.
Difference from Traditional A/B Testing
Traditional A/B testing carries the risk of showing incomplete designs to some customers during the testing period. Also, for sites with low traffic, it can take a long time to achieve statistical significance, leading to lost opportunities. With SimGym, you can complete tests in a virtual AI environment before deploying to production, allowing for verification without negatively impacting actual customers.
SimGym Plans and Pricing
SimGym is free to install, but simulations are run on a credit-based pay-as-you-go system. The pricing structure is as follows:
How Pricing Works
- Installation Fee: Free
- Initial Free Credits: 5 free simulation credits are provided upon app installation.
- Credit Consumption: Theme analysis (live theme analysis) consumes 1 credit, and comparing two themes consumes 2 credits.
- Additional Purchases: After using up your credits, additional credits can be purchased for $10 per credit.
Points to Note When Using
- Admin Screen is in English: SimGym's admin screen and AI shopper feedback are primarily displayed in English. Since Japanese support is not yet available, we recommend using a translation tool in conjunction.
- Research Preview Version: As of May 2026, it is in the research preview (beta) stage. The accuracy of simulation results should be verified independently.
- Beware of Wasting Credits: "Just running it to see what happens" can lead to wasted credits. It's important to formulate a hypothesis before running a simulation.
Trying SimGym: A Hands-On Review
From here, I'll report on my experience actually using SimGym to perform theme analysis and theme comparison (A/B testing) for a store. I'll explain the operation flow and the results obtained in detail, with accompanying images.
Checking the Admin Panel
Once SimGym is installed, you can access it from the app list in your Shopify admin panel. The top of the admin panel clearly shows the simulation mechanism in 3 steps. In the upper right corner of the screen, you'll find the remaining count of "5 free simulations" and the "Create simulation" button.
Creating and Starting a Simulation
Clicking "Create simulation" brings up a modal to select the simulation type. This time, I first selected "Analyze theme." I named it "Live Theme Analysis" and checked all four recommended focus areas: "Homepage," "Product pages," "Collections," and "Cart."
In cases where you only want to analyze product pages, for example, you would only check "Product pages."
Next, select the theme to be analyzed. Choose the desired theme from the theme selection dropdown.
This time, I will select the live theme.
Once the settings are complete, click the "Start simulation" button at the bottom of the screen. For theme analysis, the simulation cost is 1 credit.
During Theme Analysis
Once the simulation starts, the status changes to "Active" and the AI shoppers begin navigating the store. A progress bar allows you to monitor the analysis in real-time. This analysis was completed in about 10 minutes.
Even during the analysis, feedback from AI shoppers was displayed in real-time. I observed an AI shopper named "Adeline" examining products and exploring categories.
Reviewing Analysis Results
Upon completion of the analysis, improvement suggestions categorized into four areas were displayed. The details are as follows:
- Recommendations: Remove/defer cart-level terms and conditions checkbox, enhance intent-based entry points, etc.
- Site Navigation: Fix top-level menu links, clarify categorization and localization, etc.
- Product Discovery: Fix search relevance for Japanese, create filtered collections, display inventory status, etc.
- Trust Signals: Display review star ratings, integrate press mentions/certification badges, clearly state shipping policies, etc.
AI Shopper Feedback
Below the analysis results, you can review the feedback from each AI shopper. Each AI shopper has a different persona; the researcher-type "Adeline" thoroughly investigated products and explored multiple categories.
The AI Shopper list screen displays the results of multiple AI shoppers viewing different pages and products. You can see at a glance the results of diverse shopper personas such as Noa, Alicia, Samantha, Marshall, Clayton, and Kristina navigating the store.
Detailed AI Shopper Analysis
Clicking on each AI shopper will take you to their detailed analysis page. For example, "Noa" is defined as "a shopper looking for premium baby travel gear who quickly checks options and makes decisions."
On the detailed screen, quantitative data such as total time spent (1 minute 52 seconds), clicks (10 times), pages viewed (6 pages), and cart total (¥0.00) are displayed. Furthermore, screenshots of the actual pages viewed by the AI shopper and AI session insights can also be reviewed.
The activity log chronologically records how the AI shopper acted on each page and what they thought. With the message "I thought for a moment...", their thought process as they checked product specifications and materials can be observed.
The AI Session Insights present the analysis results in a more structured format. They are organized into sections: "What went well," "What didn't go well," "Outcomes," and "Recommendations," making concrete improvement actions clear.
For example, in this case, while "product navigation from categories generally worked well" was a positive point, areas for improvement were also pointed out, such as "couldn't navigate to strollers with a single click" and "material and product details were hidden within an accordion."
Both instances where the above issues occurred were intentionally implemented with accordions.
It seems that implementations using accordions are negatively rated by SimGym because they increase the number of steps for the customer.
Furthermore, the detailed analysis results show specific improvement suggestions for each category. For example, actionable items such as "remove cart-level terms and conditions checkbox" and "enhance homepage with intent-based entry points and trust signals" are provided.
The "cart-level terms and conditions checkbox" is a checkbox placed before the button with the intention of confirming that the user has read and agreed to the terms of use before checkout. The purpose is to confirm the understanding and agreement of the terms of use before purchase, which can help with customer service later.
This is a common practice in Japanese e-commerce, but it is negatively evaluated. Therefore, it's worth noting that SimGym may not provide effective analysis for measures unique to the Japanese commercial environment.
Leveraging Analysis Results with Gemini & Sidekick
SimGym's analysis results can be even more effective when combined with other AI tools. This time, I tried using the analysis results so far to summarize them with Gemini, and then used that information to request theme adjustments from Shopify's Sidekick.
Gemini summarized three high-priority improvement items as "recommended actions you can take now":
- Turn on announcement bar auto-rotation → Efficiently convey free shipping and campaign information
- Turn on predictive search → Immediate improvement of search UX
- Turn on language selector → Visibility of multilingual support
When I requested these from Sidekick, it responded with "I'll set up all three for you!" and automatically executed the theme customizations. This sequence of SimGym → Gemini → Sidekick allows for a smooth process from analysis to implementation of improvements.
Please note that since the theme being analyzed this time is Dawn-based, predictive search in Japanese does not work due to specifications, but predictive search was enabled in the settings.
While it might seem a bit mischievous, I'm excited to see how this change will affect the next theme analysis.
Comparative Analysis of Two Themes (A/B Testing)
Next, I tried SimGym's other feature, "Compare two themes." This time, I compared the original theme (for the 3/4 release) with the theme adjusted by Sidekick earlier (for SimGym verification). The simulation cost for comparative analysis is 2 credits.
After approximately 26 minutes of analysis, surprising results emerged.
The analysis showed that the add-to-cart rate for the variant theme (the one adjusted with Sidekick) was 45.57% higher.
A summary stating, "The variant performed better overall than the current theme," was displayed.
Recommendations and successes for each theme are also displayed in parallel.
Even in theme comparison, similar to theme analysis, you can check shopper feedback and detailed insights for each AI shopper. The number of clicks, pages viewed, and cart total for each AI shopper are displayed in a comparative format, allowing for a clear understanding of how user behavior differs due to theme changes.
As you may have noticed after reading this far, there was no mention of the adjustments made by SideKick earlier in the "What went well" section of the theme comparison. However, the add-to-cart rate still increased by 45%. The number of AI shoppers analyzed for each theme was around 150, which is a statistically unacceptable level of fluctuation. I examined the behavior of AI shoppers who added items to their cart for each theme, but no significant behavioral changes due to theme differences were observed. Therefore, the 45% increase in add-to-cart rate is likely due to the characteristics of the AI analyzing that particular theme. For this reason, it is recommended to treat the add-to-cart rate as a reference only. It is also recommended to use other analysis results based on the understanding that they are AI simulations, and to verify their effectiveness with actual store data.
Summary
In this article, we've provided a detailed overview and practical review of Shopify's official AI simulation app, "SimGym." Let's recap the key points:
- SimGym is an app that allows AI shoppers to virtually navigate your store, enabling you to test the effects of changes before they go live.
- It's free to install and comes with 5 free credits, making it easy to try out.
- Theme analysis provides improvement suggestions across four categories: Recommendations, Site Navigation, Product Discovery, and Trust Signals.
- Comparative analysis of two themes (A/B testing) allows for quantitative comparison of changes in add-to-cart rates.
- By combining it with Gemini and Sidekick, you can efficiently manage the PDCA cycle of analysis → improvement → re-verification.
- AI research is likely to not evaluate measures specific to the Japanese commercial environment.
- Currently, it's in the research preview phase, so it's important to use the analysis results in conjunction with actual data, rather than taking them at face value.
SimGym is a powerful tool for achieving data-driven store improvements rather than relying solely on intuition and experience. Although it's still in the research preview stage, you can easily try it out with the 5 free credits. Why not start with theme analysis?
Consider utilizing SimGym to accelerate and ensure your store improvement PDCA cycle.
も見てみる
