I applied Karpathy's autoresearch to a mobile UX and its “excellence” score went from 54% to 100%

The problem was quite simple. I have a planning feature inside almirant.ai that works well on desktop, but on mobile it never quite felt right.

It was not broken, technically everything worked, but there was too much friction; it felt like it could be simplified with small adjustments: slightly misaligned buttons, flows that required an extra tap, layouts that were not as clean as they should be. The kind of problems that are easy to tolerate but hard to prioritize.

I already had a list of fixes prepared, but instead of going through the UI manually and “polishing things”, I wanted to try something different.

The idea was to treat UX improvement as a loop, not as a series of isolated decisions. Measure the current experience, identify the weakest point, apply a change, measure again, and keep only what actually improves the result.

The first step was defining what “better” really means. Before touching anything, I created a scorecard with several areas: functionality, mobile UX quality, visual/layout quality, responsiveness, and robustness. Each one had a weight and concrete criteria behind it.

To define that scorecard I did not start from theory alone. I used the list of mobile problems I had already identified as concrete examples of things the system needed to detect and improve. In that sense, Claude Code helped me a lot: not only with implementation, but with turning vague UX problems into something much more structured and measurable.

More specifically, the system evaluated things like:

• Functionality (30 points): whether the session completed end to end, whether questions advanced correctly, whether personalized answers worked, and whether messages sent during processing were handled properly.

• Mobile UX quality (25 points): tap target size, input legibility, spacing, horizontal overflow, keyboard visibility, and screenshot-based evaluation by the LLM.

• Visual/layout quality (20 points): whether the “Next” button was always visible, whether the stepper made sense, whether there were overlaps, and whether the chat kept scroll behavior correctly.

• Responsiveness (15 points): time to interactive, transition speed between questions, streaming fluidity, and recovery after idle states.

• Robustness (10 points): resistance to double taps, reconnection, screen rotation, and flow stability under odd edge cases.

Once all that was defined, I automated the loop. I weighted the areas this way because I did not want the system optimizing for a “pretty UI” while silently breaking the real flow. The goal was simple: after each iteration, the system needed to be able to assign a number to the experience and say whether it was improving or not.

Evolución de la puntuación total de la "excelencia en mobile UX" con cada iteración del loop

Each iteration runs the full planning flow on mobile, takes screenshots at key moments, evaluates the UI both programmatically and with an LLM, and generates a score. The model identifies the weakest area, proposes a change, applies it, and runs the whole flow again. If the score improves, the change stays. If not, it is reverted.

What is interesting about this approach is that it removes a lot of subjectivity. Instead of thinking “this feels better”, you have a system constantly trying to optimize a defined surface. Things like tap target size, button visibility, scroll behavior, or overlaps stop being vague complaints and become constraints that pass or fail.

The result was not a big redesign. It was a sequence of small improvements that accumulated: fixing the visibility of the Next button, improving tap targets, making inputs more legible, handling edge cases like rotations or rapid interactions better. Each individual change is boring, but together they move the experience quite a lot.

After 20 iterations, the mobile UX score went from 54% to 100%. Not because the model “designed something brilliant”, but because it removed friction step by step, without introducing regressions.

The main lesson: do not ask AI to design things. Ask it to optimize them. Define what “better” means, let the loop run, and keep only what moves the metric. It feels more like training than a design session. And it works.

I do not think this replaces product intuition, but it does give you a way to formalize it and make it actionable. More importantly, it creates a loop where the product can improve much more systematically than by simply making manual tweaks.