Skip to content

JSONL Output Modes Implementation Plan

For agentic workers: REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Separate JSONL output from forgeview spawning so FORGE Studio can consume JSONL without a TUI, and allow JSONL output to be directed to stderr, stdout, or a file.

Architecture: A global PG_JSONL_DEST() accessor provides the FILE* handle for all JSONL output (spdlog sink + 7 direct emitter sites). A new PG_TUI_SPAWN_MODE() controls forgeview spawning independently from JSONL mode. Three new CLI flags (--no-tui, --no-jsonl, --jsonl-output) replace the old --no-tui semantics.

Tech Stack: C++17, spdlog, Boost.Program_options, Catch2

Spec: docs/superpowers/specs/2026-03-16-jsonl-output-modes-design.md

Chunk 1: Core ForgeLog.hpp Changes

Task 1: Add PG_JSONL_DEST() and PG_TUI_SPAWN_MODE() globals

Files: - Modify: forge/Core/ForgeLog.hpp

  • [ ] Step 1: Add PG_JSONL_DEST() after PG_TUI_MODE() (after line 68)
/// @brief Query or set the JSONL output destination.
///
/// Default is stderr. Set to stdout or a FILE* opened from a path.
/// All JSONL-emitting code (spdlog sink + direct emitters) uses this.
inline FILE*& PG_JSONL_DEST(FILE* set_value = nullptr, bool do_set = false)
{
    static FILE* dest = stderr;
    if (do_set)
        dest = set_value;
    return dest;
}

/// @brief Query or set the forgeview spawn mode.
///
/// Default is true (spawn enabled). Set to false via --no-tui to disable
/// forgeview spawning while keeping JSONL output active.
inline bool& PG_TUI_SPAWN_MODE(bool set_value = true, bool do_set = false)
{
    static bool mode = true;
    if (do_set)
        mode = set_value;
    return mode;
}
  • [ ] Step 2: Build to verify

Run: cmake --build build --target cpu_tests -j4 2>&1 | tail -5 Expected: builds successfully

  • [ ] Step 3: Commit
git add forge/Core/ForgeLog.hpp
git commit -m "feat: add PG_JSONL_DEST and PG_TUI_SPAWN_MODE globals"

Task 2: Parameterize jsonl sink to use PG_JSONL_DEST()

Files: - Modify: forge/Core/ForgeLog.hpp

  • [ ] Step 1: Update jsonl_stderr_sink to use PG_JSONL_DEST()

In jsonl_stderr_sink::sink_it_() (line ~165), change: 

std::fwrite(payload.data(), 1, payload.size(), stderr);
std::fflush(stderr);
to: 
std::fwrite(payload.data(), 1, payload.size(), PG_JSONL_DEST());
std::fflush(PG_JSONL_DEST());

In jsonl_stderr_sink::flush_() (line ~169), change: 

void flush_() override { std::fflush(stderr); }
to: 
void flush_() override { std::fflush(PG_JSONL_DEST()); }

Rename the class from jsonl_stderr_sink to jsonl_sink (and the alias from jsonl_stderr_sink_mt to jsonl_sink_mt) since it is no longer stderr-specific. Update the two references: - The using alias at line ~172: jsonl_stderr_sink_mt → jsonl_sink_mt - The FORGE_LOG_INIT reference at line ~427: jsonl_stderr_sink_mt → jsonl_sink_mt (Note: Task 3 fully replaces FORGE_LOG_INIT, so this reference is superseded there.)

  • [ ] Step 2: Update all 7 direct JSONL emitter sites

Each of these sites has std::fwrite(..., stderr) and std::fflush(stderr). Change each to use PG_JSONL_DEST():

  1. FORGE_TUI_START (line ~501-502): std::fwrite(msg.data(), 1, msg.size(), PG_JSONL_DEST()); std::fflush(PG_JSONL_DEST());

  2. FORGE_TUI_EXIT (line ~523-524): same pattern

  3. PGProgressManager::add (line ~603-604): same pattern

  4. PGProgressManager::tick (line ~623-624): same pattern

  5. PGProgressManager::done (line ~652-653): same pattern

  6. PGProgressManager::metrics (line ~671-672): same pattern

  7. FORGE_IMAGE_PREVIEW (line ~988-989): same pattern

  8. [ ] Step 3: Build and run tests

Run: cmake --build build --target cpu_tests -j4 && ./build/cpu_tests '~[Benchmark]' 2>&1 | tail -3 Expected: all tests pass (default dest is still stderr, behavior unchanged)

  • [ ] Step 4: Commit
git add forge/Core/ForgeLog.hpp
git commit -m "refactor: route all JSONL output through PG_JSONL_DEST()"

Task 3: Update FORGE_LOG_INIT signature

Files: - Modify: forge/Core/ForgeLog.hpp

  • [ ] Step 1: Change FORGE_LOG_INIT signature and implementation

Replace the current FORGE_LOG_INIT (lines ~411-440) with:

/// @brief Initialise the forge logger.
///
/// @param level_str     Log level string ("debug", "info", "warn", "error").
/// @param log_path      Path to the rotating log file (always human-readable).
/// @param no_jsonl      If true, disable JSONL and use classic spdlog+indicators.
/// @param jsonl_output  JSONL destination: "stderr" (default), "stdout", or file path.
inline void FORGE_LOG_INIT(const std::string& level_str = "info",
    const std::string& log_path = "forge.log", bool no_jsonl = false,
    const std::string& jsonl_output = "stderr")
{
    // Set TUI mode (JSONL active?)
    PG_TUI_MODE(!no_jsonl, true);

    // Set JSONL destination
    if (!no_jsonl) {
        if (jsonl_output == "stdout") {
            PG_JSONL_DEST(stdout, true);
        } else if (jsonl_output != "stderr") {
            // File path
            FILE* f = std::fopen(jsonl_output.c_str(), "w");
            if (f) {
                PG_JSONL_DEST(f, true);
            } else {
                // Fall back to stderr if file can't be opened
                PG_JSONL_DEST(stderr, true);
                // Can't use FORGE_WARN yet (logger not initialized), defer warning
            }
        } else {
            PG_JSONL_DEST(stderr, true);
        }
    }

    auto level = spdlog::level::from_str(level_str);

    // File sink — always human-readable
    auto file_sink = std::make_shared<spdlog::sinks::rotating_file_sink_mt>(
        log_path, 10UL * 1024UL * 1024UL, 3);
    file_sink->set_level(spdlog::level::debug);

    std::vector<spdlog::sink_ptr> sinks;

    if (PG_TUI_MODE()) {
        auto jsonl = std::make_shared<pg_detail::jsonl_sink_mt>();
        jsonl->set_level(level);
        sinks = {jsonl, file_sink};
    } else {
        auto stderr_sink = std::make_shared<spdlog::sinks::stderr_color_sink_mt>();
        stderr_sink->set_level(level);
        sinks = {stderr_sink, file_sink};
    }

    auto logger = std::make_shared<spdlog::logger>(
        "powergrid", sinks.begin(), sinks.end());
    logger->set_level(spdlog::level::debug);
    spdlog::set_default_logger(logger);
}
  • [ ] Step 2: Build and run tests

Run: cmake --build build --target cpu_tests -j4 && ./build/cpu_tests '~[Benchmark]' 2>&1 | tail -3 Expected: all tests pass (callers still pass old no_tui bool which maps to no_jsonl)

Note: The old callers pass vm["no-tui"].as<bool>() as the 3rd arg which was no_tui. This still compiles because the parameter name changed but the type is still bool. The semantic change is intentional — old --no-tui now only disables JSONL (matching the new --no-jsonl meaning). The executables will be updated in Chunk 2.

  • [ ] Step 3: Commit
git add forge/Core/ForgeLog.hpp
git commit -m "feat: update FORGE_LOG_INIT with no_jsonl and jsonl_output params"

Task 4: Update FORGE_TUI_START and FORGE_TUI_EXIT

Files: - Modify: forge/Core/ForgeLog.hpp

  • [ ] Step 1: Update FORGE_TUI_START

Replace the current FORGE_TUI_START (lines ~483-508) with:

inline void FORGE_TUI_START(
    const std::string& app_name,
    const std::string& version = FORGE_VERSION_STRING)
{
    // Case 1: JSONL disabled (--no-jsonl) — classic start log
    if (!PG_TUI_MODE()) {
        FORGE_INFO("Starting {} v{}", app_name, version);
        return;
    }

    // Case 2: JSONL enabled, spawn disabled (--no-tui) — emit JSONL start, no forgeview
    if (!PG_TUI_SPAWN_MODE()) {
        auto msg = fmt::format(
            R"({{"type":"start","app":"{}","version":"{}"}})",
            pg_detail::json_escape(app_name),
            pg_detail::json_escape(version));
        msg += '\n';
        std::fwrite(msg.data(), 1, msg.size(), PG_JSONL_DEST());
        std::fflush(PG_JSONL_DEST());
        return;
    }

    // Case 3: JSONL enabled, spawn enabled (default) — try to spawn forgeview
    std::string forgeview_path = pg_detail::find_forgeview();
    auto& tui = pg_detail::TuiProcess::instance();
    bool spawned = false;
    if (!forgeview_path.empty()) {
        spawned = tui.spawn(forgeview_path);
    }

    if (spawned) {
        auto msg = fmt::format(
            R"({{"type":"start","app":"{}","version":"{}"}})",
            pg_detail::json_escape(app_name),
            pg_detail::json_escape(version));
        msg += '\n';
        std::fwrite(msg.data(), 1, msg.size(), PG_JSONL_DEST());
        std::fflush(PG_JSONL_DEST());
    } else {
        // forgeview not available — fall back to classic
        FORGE_LOG_REINIT_CLASSIC();
        FORGE_INFO("Starting {} v{}", app_name, version);
    }
}
  • [ ] Step 2: Update FORGE_TUI_EXIT

Replace the current FORGE_TUI_EXIT (lines ~515-530) with:

inline void FORGE_TUI_EXIT(int code)
{
    auto& tui = pg_detail::TuiProcess::instance();

    // Case 1: JSONL disabled — classic log
    if (!PG_TUI_MODE()) {
        FORGE_INFO("Reconstruction finished (exit code {})", code);
        return;
    }

    // Emit JSONL exit event
    auto msg = fmt::format(R"({{"type":"exit","code":{}}})", code);
    msg += '\n';
    std::fwrite(msg.data(), 1, msg.size(), PG_JSONL_DEST());
    std::fflush(PG_JSONL_DEST());

    // Case 2: forgeview active — wait for user to press 'q'
    if (tui.active) {
        tui.wait();
    }

    // Close file handle if JSONL went to a file (not stderr/stdout)
    FILE* dest = PG_JSONL_DEST();
    if (dest != stderr && dest != stdout) {
        spdlog::shutdown(); // flush spdlog before closing the file handle
        std::fclose(dest);
        PG_JSONL_DEST(stderr, true); // reset to stderr
    }
}
  • [ ] Step 3: Build and run tests

Run: cmake --build build --target cpu_tests -j4 && ./build/cpu_tests '~[Benchmark]' 2>&1 | tail -3 Expected: all tests pass

  • [ ] Step 4: Commit
git add forge/Core/ForgeLog.hpp
git commit -m "feat: update TUI_START/TUI_EXIT for three-mode operation"

Chunk 2: Executable Integration + Version Bump

Task 5: Update forgeSense.cpp

Files: - Modify: apps/forgeSense.cpp

  • [ ] Step 1: Update CLI flags

In desc.add_options() chain, replace: 

("no-tui", po::bool_switch()->default_value(false), "Disable JSONL/TUI output, use classic spdlog+indicators")
with: 
("no-tui", po::bool_switch()->default_value(false),
 "Disable forgeview spawning (JSONL output continues on stderr)")
("no-jsonl", po::bool_switch()->default_value(false),
 "Disable JSONL output, use classic spdlog+indicators")
("jsonl-output", po::value<std::string>()->default_value("stderr"),
 "JSONL output destination: stderr (default), stdout, or file path. Implies --no-tui.")

  • [ ] Step 2: Update FORGE_LOG_INIT call

Replace the current init block (around line 123): 

FORGE_LOG_INIT(vm["log-level"].as<std::string>(), "forge.log", vm["no-tui"].as<bool>());
with: 
bool no_jsonl = vm["no-jsonl"].as<bool>();
std::string jsonl_output = vm["jsonl-output"].as<std::string>();
bool no_tui = vm["no-tui"].as<bool>();
bool warn_jsonl_ignored = false;

// --jsonl-output (non-default) implies --no-tui
if (jsonl_output != "stderr") {
    no_tui = true;
}
// --no-jsonl implies --no-tui
if (no_jsonl) {
    no_tui = true;
    if (jsonl_output != "stderr") {
        warn_jsonl_ignored = true;
    }
}

FORGE_LOG_INIT(vm["log-level"].as<std::string>(), "forge.log", no_jsonl, jsonl_output);
PG_TUI_SPAWN_MODE(!no_tui, true);

if (warn_jsonl_ignored) {
    FORGE_WARN("--jsonl-output ignored because --no-jsonl is set");
}

  • [ ] Step 3: Build and verify

Run: cmake --build build --target forgeSense -j4 2>&1 | tail -5

  • [ ] Step 4: Commit
git add apps/forgeSense.cpp
git commit -m "feat(forgeSense): new --no-tui/--no-jsonl/--jsonl-output flags"

Task 6: Update forgePcSense.cpp

Files: - Modify: apps/forgePcSense.cpp

Same pattern as Task 5.

  • [ ] Step 1: Update CLI flags (replace --no-tui definition, add --no-jsonl, --jsonl-output)
  • [ ] Step 2: Update FORGE_LOG_INIT call (same block as Task 5)
  • [ ] Step 3: Build and verify
  • [ ] Step 4: Commit
git add apps/forgePcSense.cpp
git commit -m "feat(forgePcSense): new --no-tui/--no-jsonl/--jsonl-output flags"

Task 7: Update forgePcSenseTimeSeg.cpp

Files: - Modify: apps/forgePcSenseTimeSeg.cpp

Same pattern as Task 5.

  • [ ] Step 1-4: Same as Task 6.
git add apps/forgePcSenseTimeSeg.cpp
git commit -m "feat(forgePcSenseTimeSeg): new --no-tui/--no-jsonl/--jsonl-output flags"

Task 8: Update oscillateRecon.cpp

Files: - Modify: apps/lowRank/oscillateRecon.cpp

Same pattern as Task 5.

  • [ ] Step 1-4: Same as Task 6.
git add apps/lowRank/oscillateRecon.cpp
git commit -m "feat(oscillateRecon): new --no-tui/--no-jsonl/--jsonl-output flags"

Task 9: Version bump

Files: - Modify: CMakeLists.txt

  • [ ] Step 1: Update version

In CMakeLists.txt line 9, change: 

project("forge" VERSION 0.9.0 LANGUAGES CXX)
to: 
project("forge" VERSION 0.10.0 LANGUAGES CXX)

  • [ ] Step 2: Build to verify version propagates

Run: cmake -B build -S . -DMETAL_COMPUTE=ON -DOPENACC_GPU=OFF 2>&1 | head -5 Then: cmake --build build --target forgeSense -j4 && ./build/forgeSense --version 2>&1 Expected: FORGE v0.10.0 (forgeSense) built ...

  • [ ] Step 3: Commit
git add CMakeLists.txt
git commit -m "chore: bump version to 0.10.0"

Chunk 3: Testing, Documentation, and Verification

Task 10: Add CLI tests for new flags

Files: - Modify: forge/Tests/CLITests.cpp

  • [ ] Step 1: Update existing tests that use --no-tui

Existing tests at lines ~141 and ~153 use --no-tui to suppress forgeview. Under the new semantics, --no-tui keeps JSONL. These tests only check exit codes, so they should still pass. However, update both tests to use --no-jsonl for cleaner output:

// Line ~141: "--no-tui -i /nonexistent.h5" → "--no-jsonl -i /nonexistent.h5"
// Line ~153: "--no-tui -i /nonexistent.h5" → "--no-jsonl -i /nonexistent.h5"
  • [ ] Step 2: Add missing includes to CLITests.cpp

Add at the top of forge/Tests/CLITests.cpp (after existing includes): 

#include <sstream>
#include <fstream>

  • [ ] Step 3: Add new flag interaction tests

Add test cases:

TEST_CASE("forgeSense --no-tui keeps JSONL output on stderr", "[CLI]")
{
    SKIP_IF_NO_BINARY(FORGE_SENSE_PATH);
    auto [code, output] = run_command(
        std::string(FORGE_SENSE_PATH) +
        " --no-tui -i /nonexistent_input.h5 -o /tmp/ -F NUFFT");
    // Should exit with error (missing file) but stderr should contain JSONL
    REQUIRE(code != 0);
    bool has_jsonl = false;
    std::istringstream iss(output);
    std::string line;
    while (std::getline(iss, line)) {
        if (!line.empty() && line[0] == '{') {
            has_jsonl = true;
            break;
        }
    }
    REQUIRE(has_jsonl);
}

TEST_CASE("forgeSense --no-jsonl disables JSONL output", "[CLI]")
{
    SKIP_IF_NO_BINARY(FORGE_SENSE_PATH);
    auto [code, output] = run_command(
        std::string(FORGE_SENSE_PATH) +
        " --no-jsonl -i /nonexistent_input.h5 -o /tmp/ -F NUFFT");
    // Should exit with error (missing file) but use classic output (no JSONL)
    REQUIRE(code != 0);
    // Output should NOT contain JSONL (no lines starting with '{')
    bool has_jsonl = false;
    std::istringstream iss(output);
    std::string line;
    while (std::getline(iss, line)) {
        if (!line.empty() && line[0] == '{') {
            has_jsonl = true;
            break;
        }
    }
    REQUIRE_FALSE(has_jsonl);
}

TEST_CASE("forgeSense --jsonl-output writes to file", "[CLI]")
{
    SKIP_IF_NO_BINARY(FORGE_SENSE_PATH);
    std::string tmpfile = "/tmp/forge_test_jsonl_output.jsonl";
    std::remove(tmpfile.c_str());
    auto [code, output] = run_command(
        std::string(FORGE_SENSE_PATH) +
        " --jsonl-output " + tmpfile +
        " -i /nonexistent_input.h5 -o /tmp/ -F NUFFT");
    // Check file was created and contains JSONL
    std::ifstream ifs(tmpfile);
    bool file_exists = ifs.is_open();
    bool has_jsonl = false;
    if (file_exists) {
        std::string line;
        while (std::getline(ifs, line)) {
            if (!line.empty() && line[0] == '{') {
                has_jsonl = true;
                break;
            }
        }
    }
    std::remove(tmpfile.c_str());
    REQUIRE(file_exists);
    REQUIRE(has_jsonl);
}
  • [ ] Step 4: Build and run CLI tests

Run: cmake --build build --target cpu_tests -j4 && ./build/cpu_tests "[CLI]" 2>&1 | tail -10 Expected: all CLI tests pass

  • [ ] Step 5: Run full test suite

Run: ./build/cpu_tests '~[Benchmark]' 2>&1 | tail -3 Expected: all tests pass

  • [ ] Step 6: Commit
git add forge/Tests/CLITests.cpp
git commit -m "test: add CLI tests for --no-jsonl and --jsonl-output flags"

Task 11: Update documentation

Files: - Modify: README.md - Modify: CHANGELOG.md - Modify: CLAUDE.md

  • [ ] Step 1: Update README.md

In the TUI monitoring section, update the --no-tui example and add new flags:

### Headless JSONL output (for FORGE Studio)

To get JSONL output without spawning forgeview:

```shell
forgeSense [args] --no-tui

To write JSONL to a file:

forgeSense [args] --jsonl-output /path/to/output.jsonl

To disable JSONL entirely and use classic text output:

forgeSense [args] --no-jsonl

- [ ] **Step 2: Update CHANGELOG.md**

Add under `[Unreleased]` (or create `[v0.10.0]` heading):

```markdown
## [v0.10.0] 2026-03-16

### Breaking Changes
- `--no-tui` now only disables forgeview spawning; JSONL output remains active.
  Use `--no-jsonl` to disable JSONL entirely (previous `--no-tui` behavior).

### Added
- `--no-jsonl` flag: disables JSONL output, uses classic spdlog+indicators.
- `--jsonl-output <dest>` flag: redirect JSONL to stderr (default), stdout, or a file.
  Implies `--no-tui` when destination is not stderr.
- `PG_JSONL_DEST()` global accessor for configurable JSONL output destination.
- `PG_TUI_SPAWN_MODE()` global for independent forgeview spawn control.

  • [ ] Step 3: Update CLAUDE.md

In the TUI Monitoring section, update the flag descriptions to match new semantics. Update the --no-tui description from "disable JSONL and forgeview" to "disable forgeview spawning (JSONL continues)". Add --no-jsonl and --jsonl-output.

  • [ ] Step 4: Commit
git add README.md CHANGELOG.md CLAUDE.md
git commit -m "docs: document v0.10.0 JSONL output mode changes"

Task 12: Final verification

  • [ ] Step 1: Run clang-format
find forge apps tools \( -name "*.h" -o -name "*.hpp" -o -name "*.cpp" \) ! -name "*.mm" -print0 | xargs -0 clang-format -i

Verify: find forge apps tools \( -name "*.h" -o -name "*.hpp" -o -name "*.cpp" \) ! -name "*.mm" -print0 | xargs -0 clang-format --dry-run --Werror 2>&1 Expected: no output

  • [ ] Step 2: Commit formatting if needed
git add -u && git commit -m "style: apply clang-format"
  • [ ] Step 3: Full test run

cmake --build build --target cpu_tests --target metal_tests -j4 2>&1 | tail -5
./build/cpu_tests '~[Benchmark]' 2>&1 | tail -3
./build/metal_tests '~[Benchmark]' 2>&1 | tail -3
Expected: all pass

  • [ ] Step 4: Verify version

./build/forgeSense --version 2>&1
Expected: FORGE v0.10.0 (forgeSense) built ...