hyperfine

I'm opening this ticket to discuss a potential change in Hyperfine's default behavior. For now, we always use an intermediate shell to run benchmarked programs. This allows users to make use of the full shell syntax. For example, you can run things like or that wouldn't be possible without a shell. The shell-spawn time is automatically subtracted by Hyperfine. But it still adds noise to the benchmark. The potential new default would be which can already be used today. This would have several advantages: There is no additional noise in the measurement from launching the shell. We run the program directly. All performance metrics/counters directly apply to the benchmarked program. We do not have to perform any calibration or subtraction to account for the intermediate shell. Not having to perform calibration saves some time (~100 ms) Disadvantages are: you need to do things like write instead of , expand environment variables yourself, etc. Or use when needed. Note that quoting is supported in no-shell mode though. So works fine. This might be potentially confusing to users who expect shell syntax to be available (all Hyperfine 1.0 users). If we simply fail, that's probably okay. But there might be cases where it's harder to debug what's going on ( passed to the program unexpanded).