TAAC (Test-As-A-Config) is a thrift-defined integration test framework for validating datacenter network products and platforms.
by facebookPython
Last 12 weeks · 159 commits
4 of 6 standards met
Summary Batch of OSS-mode improvements developed and validated against real Ixia hardware in our lab. Three areas: 1. VP4 thrift client factory — dependency-injected client factory so can create thrift connections via direct TCP in OSS, without ServiceRouter. 2. Unit test infrastructure — mechanism for running OSS unit tests inside the Docker environment, plus initial test coverage including import-drift regression tests. 3. Misc runtime fixes — bugs found while running bgpd_restart and other tests in OSS mode (Ixia, BGP, pyjq compat, validation steps). 1) VP4: OSS thrift client factory via dependency injection Enables to create thrift connections to FBOSS devices in OSS using direct TCP instead of Meta-internal ServiceRouter, via a dependency-injected client factory. 2) Unit test infrastructure + initial tests Adds a mechanism for running OSS unit tests inside the Docker environment, plus initial test coverage including import-drift regression tests. 3) Misc OSS runtime fixes from internal testing Fixes discovered while running bgpd_restart and other tests against real Ixia hardware in OSS mode. Test plan [ ] VP4 client factory: covers protocol compliance, all 6 client types, port defaults, HW agent port calculation, DI wiring [ ] Import drift: catches ungated Meta-internal imports at test time [ ] Existing runner tests updated and passing (, , ) [ ] Ixia changes validated against real hardware in OSS lab environment
Summary Add a complete OTG traffic generation backend as an alternative to the existing Keysight restpy-based Ixia integration. This enables TAAC to drive traffic via the open-source snappi/ixia-c stack using the same AbstractTrafficGenerator interface that TaacIxia already implements. This follows the proposal in https://docs.google.com/document/d/15OYcEgpqEPiyOsmy3QQKHMDCUqMVYb_WzRYFMS61g4U/edit?tab=t.77n1i1k4fbhu New files: taac/ixia/abstract_traffic_generator.py — 10-method ABC taac/ixia/otg_traffic_gen.py — idiomatic OTG backend (snappi) taac/libs/otg_traffic_generator.py — OtgTrafficGenerator subclass taac/otg_basic_l3_test_config.py — OTG_L3_FORWARDING config examples/topology/otg_l3_forwarding_{circuit,device}_info.csv Modified files: taac/ixia/taac_ixia.py — implements AbstractTrafficGenerator ABC taac/ixia/ixia.py — accessor for traffic start time taac/libs/taac_runner.py — begin/end_test_case lifecycle via ABC taac/libs/test_setup_orchestrator.py — backend dispatch taac/libs/traffic_generator.py — type broadened to ABC taac/health_checks/…/ixia_packet_loss_health_check.py — OTG compat thrift: port_location field, TrafficGeneratorBackend enum Test Plan Verified against local IXIA-C setup. Added new unit tests. Subsequent PRs to follow for verifying BGP with OTG.
Two unrelated OSS-compat fixes for the public slice. 1. Move EBB conveyor TestConfig aggregation out of the package eagerly imports every BAG002/BAG010/BAG011/BAG012/BAG013 conveyor TestConfig to build a single aggregation list. Because Python initializes a parent package before resolving any submodule attribute access, importing a single constant from (as does) runs the full — pulling in every bag-conveyor TestConfig and transitively closing a circular import via . Concrete frames from the live pytest failure (one of the 4 s; the other 3 close via the same chain): Fix: move the aggregated list to a sibling module and leave the package side-effect-free. No external consumer references the aggregation list from outside the package today; if any internal scheduler did, the new import path is . 2. Replace insecure with used / to produce paths for the / CLI args. has been deprecated since Python 2.3 — it returns a filename that's unique at the moment of creation but doesn't atomically create the file, leaving a TOCTOU window for an attacker to interfere with the path before it's opened. Fix: build the path inside (already created via in and removed by ): The -created directory is the atomically-created secure container; the output filename inside it just points at a future write target — no TOCTOU window, no file leak, no create-then-overwrite churn. The two alternatives the Python stdlib docs suggest don't fit this case directly: returns a file-like object with no accessible path on Linux (the file is unlinked immediately after creation). We need a path string to pass to , so gives us nothing usable. would only work with — removes the file when the exits, before opens it. then leaks the file after each test run. Test plan pytest collection: 4 s during collection before fix #1 → 0 errors, 125 tests passed after. Dry-run smoke (): passes. returns no matches in the repo after fix #2. Why this matters for future syncs The EBB conveyor structure is the internal-canonical layout, so the cycle will reappear on every future sync from the internal repository unless it's also fixed on the internal side.
Sits on top of #16 (gating) + #17 (build) + #18 (entry-point CLI runner) — those PRs' commits are replayed at the bottom because GitHub doesn't allow a non- base. The new content for this PR begins after them. What's new Seven generic OSS-mode bug fixes that surfaced while bringing up the ixia smoke against real hardware. Each is a one-file graceful-fallback patch against a Meta-internal-only assumption. Scoped to — no test-config or testbed-specific content. 1. * — was gated at import time but instantiated unconditionally → in OSS mode. Add a no-op stub class in the OSS branch. 2. — rejected hostnames (only accepted literal IPv4/IPv6). Return the input unchanged on — resolves DNS itself. 3. — IndexError on vlan-less ports. Filter ports without vlans out of the lookup dict instead of crashing. 4. — KeyError when the queried interface has only link-local IPv6 (no global v6). Skip the v6 build when isn't in . sentinel typed as so the existing falsy check still fires. 5. — no env-var override → secret had to land in the in-repo placeholder CSV. Check first; fall back to the CSV. 6. — only did IPv6 DNS as fallback → IPv4-only hosts silently resolved to , the chassis IP became empty, and the runner spent ~90s in retries before failing. Add IPv4 fallback. 7. * — registered the API server IP as a fake chassis when no port_config matched it. That "chassis" hung forever in . Early-return for single-chassis tests — daisy-chain is meaningless with one chassis. Verified All 7 fixes verified end-to-end via the ixia smoke ( + counter check, twice consecutive). The smoke itself is not part of this PR. Files , , , , — 5 files, ~80 lines net.
Sits on top of #16 (gating) + #17 (build) — those PRs' commits are replayed at the bottom because GitHub doesn't allow a non- base. The new content for this PR begins after them. What's new OSS entry-point CLI runner ( subpackage) — the command users run to execute TAAC tests under . — argparse layer ( / / / / / / / / etc.) — status enum (PASSED / FAILED / ERROR / TIMEOUT / SKIPPED / OMITTED / RETRIED / SETUP_FAILED / TEARDOWN_FAILED / TESTBED_FAILED / CONNECTION_FAILED / NOT_RUN) — POSIX-aligned exit codes (0 SUCCESS, 1-127 user errors, 128+ infra). and distinguish infra failures from real test regressions on exit code. / — result dataclass + JSON / JUnit XML emitters. is the single aggregation point; real / dominates infra-class signals so an infra blip doesn't mask real regressions. + — exception → status mapping. is the single source of truth (no second inline ladder). — wraps so each per-(, ) call captures its own . — ties it all together: parse args → load test configs → run → execute each → → emit results → return exit code. Single wraps the lifecycle so loop-bound resources survive across phases; enforced via . — minimal (DUMMY_STEP + RUN_SSH_COMMAND_STEP) usable as a worked example of what a user-supplied config looks like. Replaces an earlier wrapper script — the entry point now sets / itself. Canonical invocation Verified Live-device smoke against fboss101 + fboss102 under runs a DUMMY_STEP + RUN_SSH_COMMAND_STEP () playbook pair end-to-end via the canonical invocation. 4/4 PASSED, exit 0. Tests: covers parser, status enum, return codes, result formatter, exception classifier, end-to-end retry loop driven through . See for the full user-facing docs (exit codes, exception → status table, output formats, examples).
Repository: facebook/DNE-TaaC. Description: TAAC (Test-As-A-Config) is a thrift-defined integration test framework for validating datacenter network products and platforms. Stars: 1, Forks: 1. Primary language: Python. Languages: Python (98.8%), Thrift (0.8%), CMake (0.2%), Shell (0.2%). License: Apache-2.0. Open PRs: 2, open issues: 0. Last activity: 3h ago. Community health: 75%. Top contributors: pavanpatil92, meta-codesync[bot], vik-nexthop, shah-harshal-ai-dev, mloo3, Sahil-18, facebook-github-bot.