Last 12 weeks · 70 commits
4 of 6 standards met
Summary: This diff addresses two issues in the computation of inference trace: 1. The generation strategy is copied inside run_optimization_with_orchestrator --> we retrieve the traces on an unused generation strategy --> get_best_point defaults to the best raw observation on ALL obserations 2. Relevant data not filtered in the fallback option for get_best_parameters_from_model_predictions_with_trial_index Both of these individually lead to the inference trace being incorrect - the first to the best raw value of ALL trials, the second to the best predicted across ALL trials. Changes: Moved copying of generation strategy Added argument use_model_only_if_good to force model-based BPR even if model fit is bad Clearer sequencing in get_best_parameters_from_model_predictions_with_trial_index Differential Revision: D80019803
Summary: Adds a arg to that snaps values to a grid anchored at (in ), for both FLOAT and INT parameters. This is the first diff in the step_size unification stack: will subsume both the discrete-grid and limited-resolution () use cases under one knob. Next diff will add storage support. The internal DB has already been updated to include the new column. We will then migrate all current usage off and onto . We will add support for treating low-cardinality float-range parameters as discrete in , so that it is efficiently optimized over the correct grid (rather than having to use continuous optimization + rounding). At this point, we will have proper support for , so we can update the ax/api usage to leverage it, rather than resolving to . We can then deprecate and do any remaining clean-up. In this diff coexists with the existing arg (they are mutually exclusive at construction). Subsequent diffs in the stack migrate storage (JSON + SQA), transforms and utils, and the public API () to , then deprecate in favor of it. Behavior: rounds to the nearest integer and returns . It does NOT clamp to : an out-of-bounds input (e.g. a historical observation recorded outside the current bounds) snaps to the nearest grid point, which may itself be out of bounds. This mirrors the non- , which leaves out-of-bounds values in place rather than silently moving them into range — range validity is enforced by , not . Both bounds must lie on the grid: must be an integer multiple of (within ). Off-grid bounds are rejected at construction. This guarantees is itself a feasible value, so a value near the upper bound snaps to rather than to a grid point short of it. must be strictly positive, and must be integer-valued for INT parameters. accounts for : a grid-valued FLOAT reports the finite number of grid points instead of , and a grid-valued INT counts grid points rather than every integer in . defines a discrete grid but does not, by itself, force discrete acquisition optimization; how the optimizer treats the parameter depends on the grid cardinality and is determined at the generator level. Differential Revision: D107274057
Summary: D109061377 relaxed validation so that, in multi-objective optimization, an outcome constraint can be placed on an objective metric (e.g. while minimizing ) to bound the objective against its optimization direction. now returns for MOO. However, (used by ) was never updated. It unconditionally converted any single-metric constraint on an objective metric into an objective threshold. Objective thresholds can only bound in the direction of optimization, so a constraint like while minimizing was turned into an invalid threshold and tripped : UserInputError: Objective threshold on flops bounds from below but flops is being minimized. This diff fixes the conversion so a single-metric constraint on an objective metric becomes an objective threshold only when it bounds the objective in its optimization direction (upper bound on a minimized objective, lower bound on a maximized one). A constraint that bounds against the optimization direction is kept as a true outcome constraint, which MOO now supports. For with -> outcome constraint (against the minimize direction) train_ne objective threshold (aligned with minimize) Differential Revision: D109856433
Summary: Implements the feature requested in https://github.com/facebook/Ax/issues/4688: a option on the high-level API, so users can disable modeling of tracking metrics without manually editing the generation strategy. When , only metrics in the optimization config (objectives and outcome constraints) are modeled. Tracking metrics are still recorded but not modeled by the BoTorch model — useful when many tracking metrics are kept only for book-keeping. Changes — added to and , threading it into the dispatch struct. Docstrings flag that when , model-dependent analyses (cross-validation, sensitivity analysis, etc.) will not be produced for tracking metrics, that it requires an optimization config, and that it has no effect for . — added the field to . — injects into the MBM node's ; only the MBM node receives it (Sobol/random-search nodes don't model outcomes). Tests** — added (dispatch) and (client); updated existing tests that assert the full MBM dict. Implementation note is a kwarg. The registry's kwarg-splitter () automatically routes any key matching an adapter constructor param into the adapter, so adding it to the node's is sufficient — no changes to , , or . Test Plan: Dispatch tests: 13 passed. Client tests pass. flake8 + ufmt clean. End-to-end verified with a genuine tracking metric : → modeled outcomes ; → (tracking metric excluded from modeling). Closes https://github.com/facebook/Ax/issues/4688 Reviewed By: mpolson64 Differential Revision: D109735998 Pulled By: saitcakmak
Repository: facebook/Ax. Description: Adaptive Experimentation Platform Stars: 2771, Forks: 371. Primary language: Python. Languages: Python (95.2%), Jupyter Notebook (4.2%), JavaScript (0.3%), Makefile (0.1%), Batchfile (0.1%). License: MIT. Homepage: https://ax.dev Latest release: 1.3.1 (3w ago). Open PRs: 100, open issues: 83. Last activity: 16h ago. Community health: 75%. Top contributors: saitcakmak, mpolson64, lena-kashtelyan, esantorella, sdaulton, bernardbeckerman, mgarrard, ldworkin, Balandat, 2timesjay and others.