sedum
natural-language e2e

AI tests already exist. You just can't run them on every PR.

Momentic, Cypress's cy.prompt and others already run tests written in plain English. Sedum is the open-source one, built on Jev so a 50-test PR suite costs $91 a month at worst, not thousands.

One email when it's ready to install. Nothing else.

five sentences, one browser, one report 0:38
what we think

Six things about testing with AI.

01

Writing tests in English is a good idea. Somebody already had it.

It works, and people use it. We're not going to pretend we invented it.

02

A test you run twice a week isn't a safety net.

You want every test on every pull request. That's where the pricing breaks: these tools charge per step, and a modest suite blows through the plan.

03

Cheap has to mean cheap at volume.

Jev is fast and cheap enough to ask on every step of every run. That changes how many tests you can afford to write, and how often you can afford to run them.

04

Open source. No black box between your tests and the verdict.

The prompts, the scoring and the cache will all be in the repo, so you can see what goes to the model and what comes back. It runs on your own model key, on your own machine. We're finishing the release now.

05

A pass at 60% isn't a pass at 99%.

Every check comes back as a probability measured against a threshold you set, plus a second question: does the page contradict the claim? High on both is marginal, never a pass.

06

If the page moved, the report should say so.

When a cached element stops resolving, Sedum finds it again and keeps going, then grades the run DIVERGED instead of green. The test passing and the product changing are different facts.

the numbers

What a 20-person team's PR suite costs.

About a dozen engineers, one pull request a day each, pushed twice: roughly 20 runs of every test. Fifty tests, ten steps each, three of them AI steps. Priced per step, then on Jev.

the usual way

Per-step platforms

$4,875a month
$4,750 · 380,000 credits over the 10,000 included $125 · the plan
on Jev, your own key

Sedum

$91a month, worst case
$38 · low end up to $91 · worst case, cache off
a month, one linear scale
53× lessat the worst case, on the same suite: 30,000 runs a month.
Top: a per-step metered platform at its published pay-as-you-go rates: 1 credit a step, 2 for an AI step, $0.0125 a credit, 10,000 credits in the $125 plan. Its own estimator sends volumes like this to a custom Enterprise plan, so a real quote may differ. Bottom: measured with the cache off, at Jev's posted $0.042 per million input tokens (output is free). Flows on Wikipedia, Hacker News, MDN and GitHub cost $0.0013 a run; scaled to 8.4 model calls a run, $0.0030. Simple pages cost far less, and on that suite a warm cache cut model calls from 42 to 9.
the speed

Fast enough to ask on every step.

A model call per step sounds slow. On Jev it's under half a second, so a 17-step checkout finishes in 14 seconds. The same flow takes 69 on a per-step platform.

the usual way

Per-step platforms

69seconds
on Jev, your own key

Sedum

14seconds
8.5 s · browser and page 5.7 s · waiting on the model
seconds, one linear scale
4.9× fasteron the same checkout, five runs each, cache off.
One purchase flow on saucedemo.com — log in, add two items, open the cart, check out, confirm — run five times each with every cache off, on one laptop. Sedum: 13.4–14.7 s. A per-step platform's own local runner: 65–73 s. Sedum's version has 17 steps (it also opens the page and checks the confirmation); the other has 15. The model wait is the mean of five runs, across 16 calls.
what you write

Write the steps, or write the goal.

Spell out each step, or state the goal and let Sedum work out the steps. Either way it's a short YAML file.

flows/login.yaml steps 
url: https://shop.example.com/

steps:
  - type {{user}} in the username field
  - type {{password}} in the password field
  - click the login button
  - verify a list of products is shown
One line per action. No selectors.
flows/checkout.yaml goal 
url: https://shop.example.com/

goal: >
  sign in, add a hat to the cart, check out
  with the saved card

verify: the confirmation page shows
  an order number
Describe the outcome. Sedum works out the clicks.
early access

Be the first to install it.