feat(logging): add Logger interface and default logger (2/6)

[kamcheungting-db]

Part 2 of the logging stack (builds on #722). Adds the logging API and a swappable default logger — the foundation the backends and macros plug into.

What's here

• Logger: the pluggable sink interface (ShouldLog / Log / SetLevel / Flush / Initialize). ShouldLog() is the single source of truth for runtime filtering.
• LogMessage owns its formatted text so a sink can safely keep a record; reserves an attributes field for future structured logging.
• Process-global default logger: GetDefaultLogger / SetDefaultLogger / SetDefaultLevel, with a lock-free thread-local fast path so logging stays cheap.
• Initialize applies the level property, so config-driven levels actually work. CurrentLogger() is safe to call even from a thread_local destructor during thread shutdown.
• logger.h stays backend-agnostic (never includes the build config header), so consumers see one stable API regardless of backend.

Examples — using the API directly (the LOG_* macros that wrap it arrive in #725):

// A custom sink, installed as the process default.
class MySink : public Logger {
 public:
  bool ShouldLog(LogLevel level) const override { return level >= level_; }
  void Log(LogMessage&& m) noexcept override { write_line(m.message); }
  void SetLevel(LogLevel level) override { level_ = level; }
  LogLevel level() const override { return level_; }
 private:
  std::atomic<LogLevel> level_{LogLevel::kInfo};
};

SetDefaultLogger(std::make_shared<MySink>());   // install process-wide
SetDefaultLevel(LogLevel::kDebug);              // adjust the threshold

auto logger = GetDefaultLogger();               // borrow the current default
if (logger->ShouldLog(LogLevel::kInfo)) {
  logger->Log(LogMessage{.level = LogLevel::kInfo, .message = "scan ready"});
}

// Or configure from catalog-style properties (applies the "level" key):
auto sink = std::make_shared<MySink>();
auto status = sink->Initialize({{std::string(kLevelProperty), "warn"}});  // -> kWarn

The same example is documented inline in logger.h.

Tests — logger_test: default-logger API, level-from-property, invalid level rejected, concurrent swap/read, and logging during thread teardown. Built and run with clang/libc++ (spdlog ON and OFF).

This pull request and its description were written by Isaac.

[wgtmac]

Suggested change

	(void)l->ShouldLog(LogLevel::kError);
	(void)GetDefaultLogger();
	std::ignore = l->ShouldLog(LogLevel::kError);
	std::ignore = GetDefaultLogger();

[kamcheungting-db]

Done

[wgtmac]

Let's use IsError matcher.

[kamcheungting-db]

Done

[wgtmac]

Do we need to add ICEBERG_EXPORT to this and below?

[kamcheungting-db]

Done

[wgtmac]

I think this is still not safe for thread_local teardown.

One case: a thread creates another thread_local object first. Then it calls CurrentLogger() in normal code. At thread exit, alive is destroyed before that earlier object. If that earlier object logs in its destructor, we read alive.value after alive is already destroyed. This is UB.

The test only checks first use from a TLS destructor. Maybe we need a design without destructible TLS state, or leak the per-thread cache.

[kamcheungting-db]

Done.
The new thread local is implemented and guaranteed teardown logic safe.
The tear down logic is protected by a thread local Guard.

ThreadCache* AccessThreadCache(bool create) noexcept {
  static thread_local bool dead = false;
  static thread_local ThreadCache* cache = nullptr;
  static thread_local struct Guard {
    ~Guard() {
      dead = true;  // mark BEFORE freeing, so a re-entrant log hits the fallback
      delete cache;
      cache = nullptr;
    }
  } guard;
  std::ignore = guard;  // mark the thread_local as intentionally used (its dtor is
                        // registered by reaching the declaration above)

  if (dead) return nullptr;
  if (cache == nullptr && create) cache = new ThreadCache();
  return cache;
}

[wgtmac]

The doc says logger implementations must not throw. ShouldLog() is also in the hot path.

But the virtual method is not noexcept. A custom engine logger may throw here by mistake. Then logging can throw from the filter path. Adding noexcept later is also an API/ABI change.

Can we mark ShouldLog() noexcept now? Maybe SetLevel() and level() too, if we want the same rule for all logger methods.

[kamcheungting-db]

done

[wgtmac]

Small doc issue. This header says it has logging macros, but this PR does not add them.

The PR text says macros are in a later block. Users who read the installed header may be confused.

Maybe say this header has the logger API and macro helper hooks. Or move this sentence to the later PR.

[kamcheungting-db]

done.

[wgtmac]

I think only having a process-global default logger may be hard for engine integration.

In engines, users may want different loggers for different catalog, session, query, or task. If all code only reads one global logger, it is hard to route logs to the engine logging context.

Can we consider a way to pass or bind a logger from a higher level context? The global logger can still be the fallback.

[kamcheungting-db]

Done

[wgtmac]

I think we should keep a lazy format-style API here.

Now Emit takes an already formatted std::string. This means the string must be built before calling the sink. It is OK after ShouldLog(), but it still makes the API string-based. Some engine loggers may want to keep format string + args, or format only inside their own backend.

Can we provide an API closer to std::format style, like log(level, fmt, args...), and only build the string after the level check? This also makes normal use easier for callers.

[kamcheungting-db]

Done

[kamcheungting-db]

we now support:
iceberg::Log(LogLevel::kInfo, "loaded {} files", n);
and
iceberg::Log(logger, LogLevel::kWarn, "retry {}", attempt);

[wgtmac]

I think the attributes field is useful, but now there is no easy API to fill it.

For engine loggers, structured fields are often important. For example query id, task id, table name, snapshot id, file path, etc. If the first public API only supports message text in practice, it may be harder to add good structured logging later.

Can we add a small helper or builder path for attributes, or at least define how future macros will pass them?

[kamcheungting-db]

Done, we now support

auto record = LogMessage::Builder(LogLevel::kInfo)
                  .Message("scan finished")
                  .Attribute("table", table_name)
                  .Attribute("snapshot_id", std::to_string(id))
                  .Build();
logger->Log(std::move(record));