Fix startup crash: examples are cached eagerly, so an endpoint error stops the Space booting

#8
by devsinghi - opened

The Space is currently in RUNTIME_ERROR and won't start. From the traceback in #7:

gradio/routes.py    startup_events()
gradio/blocks.py    run_extra_startup_events()
gradio/helpers.py   _start_caching()      <- examples are executed at launch
gradio/helpers.py   cache()
simulation_scripts.py:279  run_md_simulation()
→ 400 Client Error: https://khy6jl8mzwonanl2.us-east-1.aws.endpoints.huggingface.cloud/
→ gradio.exceptions.Error → app never finishes starting

All six gr.Examples blocks pass cache_examples=True, and cache_mode defaults to
eager, so every example runs during startup — each one calling the remote inference
endpoint. One failing example takes down the whole Space, not just the feature that needs
the endpoint.

Fix

Add cache_mode="lazy" alongside the existing cache_examples=True. Examples are then
cached on first use instead of at launch. In gradio/helpers.py the eager path is behind
a strict identity check that "lazy" does not satisfy:

if self.cache_examples and cache_mode == "lazy":
    self.cache_examples = "lazy"      # normalised
...
if self.cache_examples is True:       # "lazy" does not match
    await self.cache()                # the startup execution that fails

Two alternatives were considered and rejected:

  • cache_examples=False fixes the crash but discards caching entirely, so every visitor
    clicking an example would trigger a fresh MD simulation and endpoint call. These
    examples are expensive, which is presumably why they were cached.
  • cache_examples="lazy" works but is deprecated — gradio warns "In future versions of
    Gradio, the cache_examples parameter will no longer accept a value of 'lazy'. You
    should set cache_examples=True, and cache_mode='lazy' instead."

Tested on gradio 5.23.3 (the version this Space pins)

Same gr.Examples pattern, with a function that raises gr.Error:

result deprecation warning
cache_examples=True (current) STARTUP FAILED
cache_examples=False LAUNCHED OK no
cache_examples="lazy" LAUNCHED OK yes
cache_examples=True, cache_mode="lazy" LAUNCHED OK no

cache_mode is present in 5.23.3, so this needs no version bump. The failing case's
traceback goes through _start_cachingcache(), the same path as the report in #7.

End-to-end, running this Space's own app.py

Real app.py on gradio 5.23.3, INFERENCE_ENDPOINT_URL pointed at a dead address to
stand in for the failing endpoint. Molecule3D substituted with gr.Model3D and the
OAuth mock stubbed, since neither can run off-Space; nothing else changed.

outcome
current app.py prints Caching examples at: .../cached_examples/32, blocks ~7 min retrying the endpoint, then Exception: Couldn't start the app because 'http://127.0.0.1:7860/gradio_api/startup-events' failed (code 500)
patched app.py boots in seconds, no caching at startup

Both queue the same 6 startup events, so the fix isn't dropping examples — it moves when
they run. The failure text is the same startup-events 500 the Space reports today.

Change

One line added after each cache_examples=True, at 208, 433, 499, 555, 716, 875. Six
lines in app.py, nothing else touched; run_on_click=True left as-is.

This does not fix the 400 itself. If the endpoint is still unhealthy a simulation will
fail when an example is clicked — but the Space loads and surfaces the error instead of
being unreachable.

devsinghi changed pull request title from Fix startup crash: cache_examples=True makes the Space fail to boot when the endpoint errors to Fix startup crash: examples are cached eagerly, so an endpoint error stops the Space booting
lbluque changed pull request status to merged

Sign up or log in to comment