> ## Documentation Index
> Fetch the complete documentation index at: https://arka-agent-mintlify-f21cea86.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing Arka with the pytest suite

> Run Arka's pytest suite locally and in CI, understand what the test coverage areas exercise, and add new tests when contributing skills or plugins.

Arka uses **pytest** for automated tests. The suite covers routing, LLM failover, vision, integrations, telemetry, and skill-specific behavior.

## Continuous integration

Pull requests and pushes to `main` run [`.github/workflows/ci.yml`](https://github.com/Sumit884-byte/arka/blob/main/.github/workflows/ci.yml):

1. **Ruff** — `ruff check src/ tests/`
2. **Pytest gate** — MCP, OpenClaw/Hermes, clipboard, agent hub, and LLM fallback suites (expand as other modules stabilize)

Tag pushes still use the separate [Publish](https://github.com/Sumit884-byte/arka/blob/main/.github/workflows/publish.yml) workflow for PyPI.

Locally you can still run the full suite with `pytest tests/`.

Docs changes should also pass the local docs reliability checker:

```bash theme={null}
python scripts/check_docs.py docs
```

## Run tests

From a dev checkout:

```bash theme={null}
cd arka
pip install -e ".[dev]"
arka "pytest tests/"
```

Run a single file or test:

```bash theme={null}
arka "pytest tests/test_llm_fallback.py"
arka "pytest tests/test_router_show_me.py -v"
arka "pytest tests/test_currency_convert.py::test_specific_case"
```

## Extreme production-readiness constraints

For critical changes, ask Arka for the opt-in high-rigor production-readiness preset:

```bash theme={null}
arka extreme constraints for checkout
arka production ready constraints for checkout
arka qa extreme --feature checkout
arka qa extreme --json
```

This expands the normal QA plan into explicit constraints for correctness, quality gates, security/privacy, reliability/operations, and release readiness. It includes unit tests, Gherkin/acceptance tests, integration and contract tests, e2e/smoke tests, mutation testing, test coverage, static analysis, secrets/privacy checks, dependency safety, observability, performance, rollback, configuration validation, backward compatibility, documentation, and migration integrity.

Arka should treat every claim as evidence-backed. If mutation testing, coverage, security scanning, SLOs, or a production metric cannot be measured in the current project, it reports that honestly instead of inventing a pass.

Install dev extras (includes `pytest` and `ruff`):

```bash theme={null}
pip install -e ".[dev]"
```

## Test suite overview

The `tests/` directory is organized by area:

### Routing and NL parsing

| Module | What it tests |
| - | - |
| `test_router_show_me.py` | "Show me" queries must not route to `describe_image` |
| `test_router_github_repo.py` | GitHub repo skill routing |
| `test_router_find_files_by_size.py` | File-size search routing |

### LLM and providers

| Module | What it tests |
| - | - |
| `test_llm_fallback.py` | Provider chain parsing, failover, per-skill models |
| `test_vllm_cloud.py` | vLLM cloud provider integration |
| `test_vllm_cross_platform.py` | vLLM on macOS/Linux/Windows |
| `test_word_limit.py` | Output word-limit enforcement |

### Vision and media

| Module | What it tests |
| - | - |
| `test_describe_image_resize.py` | Image resize before vision API calls |
| `test_describe_image_ocr_output.py` | OCR output formatting |
| `test_describe_screen.py` | Screen description skill |
| `test_compose_video_captions.py` | Video composition captions |
| `test_compose_slides.py` | Presentation slide deck generation and format conversion |
| `test_chart_fetch.py` | Chart external data fetch |

### Integrations

| Module | What it tests |
| - | - |
| `test_currency_convert.py` | Currency conversion skill |
| `test_price_check.py` | Price check skill |
| `test_github_repo.py` | GitHub repo stats |
| `test_gmail_draft.py` | Gmail draft creation |
| `test_install_app_platform.py` | Cross-platform app install |
| `test_product_reviewer.py` | Product review skill |
| `test_memory_search_fallback.py` | Memory search fallback chain |
| `test_openclaw_features.py` | OpenClaw compatibility features |
| `test_hermes_features.py` | Hermes-inspired sessions and sub-agents |

### Observability

| Module | What it tests |
| - | - |
| `test_telemetry.py` | OpenTelemetry tracing setup |
| `test_signoz_setup.py` | SigNoz configuration |
| `test_supermemory_obs.py` | Supermemory observability hooks |
| `test_mcp_obs.py` | MCP client observability |

## Writing new tests

Place tests in `tests/` with the `test_` prefix. Follow existing patterns:

```python theme={null}
import pytest

def test_my_skill_routes_correctly():
    from arka.router import route
    result = route("convert 100 USD to INR")
    assert "currency" in result.lower()
```

For routing tests, use `arka.routing.symbolic` helpers or `arka.router.route()` directly. Mock external APIs (LLM, web search) when tests should not hit the network.

## Linting

```bash theme={null}
arka "ruff check src/ tests/"
```

`ruff` is included in the `[dev]` extra.

## Local verification scripts

Beyond pytest, the repo includes helper scripts:

```bash theme={null}
python3 scripts/verify_features.py   # live feature smoke tests
python3 scripts/sync_bundled.py      # sync bundled/ before wheel build
python3 scripts/check_docs.py docs   # internal docs links + frontmatter
```

<Note>
  Some tests require optional extras (`[chat]`, `[charts]`, `[drawings]`) or API keys. Tests that need network access are typically mocked. If a test fails on import, install the matching extra first.
</Note>


## Related topics

- [Repo health](/guides/repo-health.md)
- [Self-improve loop](/guides/self-improve.md)
- [Arka CLI command and flag reference](/guides/cli.md)
- [Skills catalog: 70+ built-in Arka commands](/guides/skills.md)
- [How to code with Arka](/guides/code-with-arka.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.