Tweaks and fixes throughout the project (#27)

This PR contains the work done to do a little bit of housekeeping pass across all packages and the Website service.

To provide further details about the work:
* Refreshed the READMEs and source documentation to match the current code;
* Tagged every test case consistently across the Infrastructure, Localization, Persistence, and Website test targets;
* Removed Website middleware tests now covered by Infrastructure's own suite;
* Conformed the `PrepareDB` method to Sendable;
* Relaxes the production Compose DATABASE_TLS default from require to prefer;
* Added Persistence test verifying the prefer posture falls back to plaintext connections.

Reviewed-on: rock-n-code/loud-amsterdam#27
Co-authored-by: Javier Cicchelli <javier@rock-n-code.com>
Co-committed-by: Javier Cicchelli <javier@rock-n-code.com>
This commit is contained in:
2026-07-30 06:33:57 +00:00
committed by javier
parent 0887135328
commit ea00b94841
71 changed files with 633 additions and 512 deletions
+21 -1
View File
@@ -32,6 +32,14 @@ let package = Package(
url: "https://github.com/vapor/sql-kit.git",
from: "3.36.0"
),
.package(
url: "https://github.com/vapor/mysql-nio.git",
from: "1.7.0"
),
.package(
url: "https://github.com/apple/swift-nio.git",
from: "2.65.0"
),
],
targets: [
.target(
@@ -59,7 +67,19 @@ let package = Package(
.testTarget(
name: "PersistenceTests",
dependencies: [
.byName(name: "Persistence")
.byName(name: "Persistence"),
.product(
name: "MySQLNIO",
package: "mysql-nio"
),
.product(
name: "NIOCore",
package: "swift-nio"
),
.product(
name: "NIOPosix",
package: "swift-nio"
),
],
path: "Tests"
),
+60
View File
@@ -0,0 +1,60 @@
# Persistence
The [Fluent](https://github.com/hummingbird-project/hummingbird-fluent)-based data layer the **Loud** services build on: runtime selection between a MySQL/MariaDB backend and an ephemeral in-memory SQLite one, single-place migration registration, and a database readiness probe.
## Overview
The package provides, grouped by role:
| Role | Types |
| --- | --- |
| Backend selection | `Driver` (`mysql` or `inMemory`), `Configuration` (the MySQL/MariaDB connection parameters), `TLS` (the connection's TLS posture) |
| Service | `Service`, which builds the `Fluent` service configured for the chosen driver |
| Migrations | `PrepareDB`, the single registrar declaring every migration, in order |
| Readiness | `Probe`, which reports whether the default database answers a `SELECT 1` |
| Scaffolding (internal) | `ExampleRecord`, `CreateExampleRecord`, and `ExampleRepository` — the model → migration → repository pattern, to be replaced by the first real domain model |
## Design rules
- **The package reads no configuration.** The executable maps its `database.*` keys onto a `Driver` and hands it over; connection values arrive as plain data. See the Website service's `ConfigReader+Properties` for the mapping.
- **One default database.** `Service` registers the selected backend as the *default* database, so repositories resolve it with a plain `fluent.db()` and stay agnostic of which driver is in use.
- **Migrations are declared once, and append-only.** `PrepareDB` is the single place migrations are registered, in the order they must run; alter the schema by adding a new migration, never by editing one that has already run. Registering does not apply them — the in-memory backend is migrated on startup, while a shared MySQL/MariaDB database is migrated out of band (the executable's migrate-and-exit mode), so multiple booting instances never race.
- **Models never cross a concurrency boundary.** FluentKit models are mutable reference types; repositories map them to `Sendable` value-type snapshots (e.g. `Example`) before returning, and the models themselves stay internal to the package.
- **Readiness never throws.** `Probe` runs a `SELECT 1` — the cheapest statement both backends understand, independent of any schema — and maps every failure to `false`, so callers translate it straight into a readiness response.
- **A single connection for the in-memory store.** The SQLite backend is capped at one connection per event loop so every query reaches the same in-memory database, rather than each pooled connection getting its own private one.
- **Method structs.** `Service`, `PrepareDB`, and `Probe` hold their lifetime-fixed configuration in `init` and take only per-call inputs in `callAsFunction`.
> **Note:** the `prefer` TLS posture is enforced by the driver itself: a supplied TLS configuration upgrades the connection only when the server advertises TLS, and continues in plaintext otherwise (pinned by a test against a fake server that offers no TLS). `require` currently maps to the same configuration and therefore behaves like `prefer` — the refusal when the server offers no TLS is not yet enforced.
## Layout
Sources are split by visibility, then by kind, one type per file:
```
Sources/
├── Public/
│ ├── Enumerations/ Driver, TLS
│ ├── Methods/ Service, PrepareDB, Probe
│ └── Types/ Configuration
└── Internal/
├── Migrations/ CreateExampleRecord
├── Models/ ExampleRecord
└── Repositories/ ExampleRepository (returning the Example snapshot)
Tests/
├── Cases/ the test suites, mirroring the Sources/ layout
└── Utils/ the NotSQL* fakes backing the probe's non-SQL-database case, the
plaintext-only fake MySQL server, and the suite Tag constants
```
## Testing
The suite runs against the in-memory backend by default, so `swift test` needs no database. The MySQL/MariaDB integration test is skipped unless a database is pointed at via `MYSQL_TEST_HOST` (with optional `MYSQL_TEST_PORT`, `MYSQL_TEST_NAME`, `MYSQL_TEST_USERNAME`, and `MYSQL_TEST_PASSWORD`); it reverts its migrations afterwards, so the shared database is left as it was found:
```sh
# in-memory only
swift test
# or
# with the local MariaDB up (make db-mount):
MYSQL_TEST_HOST=127.0.0.1 swift test
```
Outside the application's service group, a built `Fluent` service must be shut down explicitly — even on failure — or its connection pool asserts on `deinit`; the suites' `do`/`catch` pattern around `fluent.shutdown()` is the shape to follow.
Every suite carries a tag naming the kind of API it exercises — `.enumeration` or `.method`, declared in `Tests/Utils/Extensions/Tag+Constants.swift` — so test plans and result summaries can slice the run by kind. A new suite must adopt the tag matching its subject (or add a tag there if none fits).
## Requirements
- Swift 6.3 toolchain (`swift-tools-version:6.3`).
- macOS 15, matching the sibling `Infrastructure` and `Localization` packages (the services deploy to Linux containers; the packages carry no UI platforms).
- Package dependencies: `hummingbird-fluent`, `fluent-mysql-driver`, `fluent-sqlite-driver`, and `sql-kit`; the test target additionally depends on `mysql-nio` and `swift-nio` for the TLS fallback test.
@@ -1,20 +1,17 @@
/// The persistence backend the service runs against.
///
/// The executable picks a driver at startup and hands it to ``Service``, which registers the
/// matching database as the default one. Repositories resolve that default and stay agnostic
/// of which backend is in use.
/// The executable picks a driver at startup and hands it to ``Service``, which registers the matching database as the default one. Repositories resolve
/// that default and stay agnostic of which backend is in use.
public enum Driver: Sendable {
/// A MySQL/MariaDB server, reached with the given connection parameters.
///
/// - Parameter configuration: the host, credentials, TLS posture, and pooling limits the
/// connection is opened with.
/// - Parameter configuration: the host, credentials, TLS posture, and pooling limits the connection is opened with.
case mysql(Configuration)
/// An ephemeral, in-process SQLite database held entirely in memory.
///
/// Nothing is written to disk, and all data is lost when the service stops intended for
/// local development and tests.
/// Nothing is written to disk, and all data is lost when the service stops intended for local development and tests.
case inMemory
}
@@ -2,9 +2,8 @@ import NIOSSL
/// The TLS posture used when connecting to the database.
///
/// The executable derives a posture from its `database.tls` configuration and passes it along as
/// part of ``Configuration``; the MySQL driver receives the resulting `TLSConfiguration` through
/// ``tlsConfiguration``.
/// The executable derives a posture from its `database.tls` configuration and passes it along as part of ``Configuration``; the MySQL driver
/// receives the resulting `TLSConfiguration` through ``tlsConfiguration``.
public enum TLS: Sendable {
/// Connect without TLS, in plaintext.
@@ -14,6 +13,9 @@ public enum TLS: Sendable {
case prefer
/// Connect only over TLS, refusing the connection when the server offers none.
///
/// - Important: the refusal is not yet enforced until it is, `require` behaves like ``prefer`` and silently falls back to plaintext when the
/// server offers no TLS.
case require
}
@@ -24,18 +26,15 @@ extension TLS {
/// The NIO TLS configuration passed to the MySQL driver for this posture.
///
/// Returns `nil` for ``off`` (connect in plaintext) and the default client configuration for
/// ``prefer`` and ``require``.
/// Returns `nil` for ``off`` (connect in plaintext) and the default client configuration for ``prefer`` and ``require``.
///
/// - Note: `prefer` and `require` currently map to the same client configuration both enable TLS.
/// The distinction (fall back to plaintext vs. fail when the server offers no TLS) is not yet
/// enforced here; tighten this mapping if that guarantee becomes required.
/// - Note: the driver gives a supplied configuration ``prefer`` semantics natively it upgrades to TLS only when the server advertises support,
/// and continues in plaintext otherwise so `prefer` is fully enforced. `require` maps to the same configuration and therefore currently
/// behaves like ``prefer``: the refusal when the server offers no TLS is not yet enforced.
var tlsConfiguration: TLSConfiguration? {
switch self {
case .off:
return nil
case .prefer, .require:
return .makeClientConfiguration()
case .off: nil
default: .makeClientConfiguration()
}
}
@@ -2,9 +2,9 @@ import HummingbirdFluent
/// A registrar declaring every migration against a `Fluent` service.
///
/// Built once around the application's `Fluent` service and called as a function `await migrate()`
/// during startup, before the migrations are applied.
public struct PrepareDB {
/// Built once around the application's `Fluent` service and called as a function `await migrate()` during startup, before the migrations are
/// applied.
public struct PrepareDB: Sendable {
// MARK: Initializers
@@ -15,9 +15,9 @@ public struct PrepareDB {
/// Registers every migration against the `Fluent` service, in order.
///
/// This is the single place migrations are declared: add each new migration here, in the order it must
/// run (migrations are applied in registration order and are append-only). Registering does not apply
/// them the caller runs `fluent.migrate()` (or the executable's migrate-and-exit mode) to do that.
/// This is the single place migrations are declared: add each new migration here, in the order it must run (migrations are applied in registration order
/// and are append-only). Registering does not apply them the caller runs `fluent.migrate()` (or the executable's migrate-and-exit mode) to do
/// that.
public func callAsFunction(
for fluent: Fluent
) async {
@@ -3,8 +3,8 @@ import SQLKit
/// A readiness probe reporting whether the database behind a `Fluent` service is reachable.
///
/// Built once around the application's `Fluent` service and called as a function whenever a fresh
/// answer is needed typically from a readiness endpoint: `let ready = await probe()`.
/// Built once around the application's `Fluent` service and called as a function whenever a fresh answer is needed typically from a readiness endpoint:
/// `let ready = await probe()`.
public struct Probe: Sendable {
// MARK: Properties
@@ -26,11 +26,10 @@ public struct Probe: Sendable {
/// Reports whether the database behind the `Fluent` service is reachable.
///
/// Runs a trivial `SELECT 1` against the default database the cheapest statement both the MySQL/MariaDB
/// and SQLite backends understand so a readiness check does not depend on any particular schema or model.
/// Any failure (connection refused, authentication error, pool exhausted) is reported as not reachable
/// rather than thrown, so callers can map it straight onto a readiness response. A default database that
/// is not an SQL database is likewise reported as not reachable.
/// Runs a trivial `SELECT 1` against the default database the cheapest statement both the MySQL/MariaDB and SQLite backends understand so
/// a readiness check does not depend on any particular schema or model. Any failure (connection refused, authentication error, pool exhausted) is
/// reported as not reachable rather than thrown, so callers can map it straight onto a readiness response. A default database that is not an SQL
/// database is likewise reported as not reachable.
/// - Returns: `true` when the database answers the probe, `false` otherwise.
public func callAsFunction() async -> Bool {
guard let database = fluent.db() as? any SQLDatabase else {
@@ -5,8 +5,7 @@ import Logging
/// A factory building the `Fluent` service the application persists through.
///
/// Built once around the driver the executable picks at startup and called as a function to produce
/// the configured service: `let fluent = service()`.
/// Built once around the driver the executable picks at startup and called as a function to produce the configured service: `let fluent = service()`.
public struct Service: Sendable {
// MARK: Properties
@@ -35,10 +34,9 @@ public struct Service: Sendable {
/// Builds a `Fluent` service configured for the driver.
///
/// The selected backend is registered as the *default* database, so repositories resolve it with a plain
/// `fluent.db()` and stay agnostic of which driver is in use. The returned service is not yet running; add
/// it to the application's service group (`app.addServices(_:)`) so it starts and shuts its connection pool
/// down alongside the server.
/// The selected backend is registered as the *default* database, so repositories resolve it with a plain `fluent.db()` and stay agnostic of which
/// driver is in use. The returned service is not yet running; add it to the application's service group (`app.addServices(_:)`) so it starts and shuts
/// its connection pool down alongside the server.
/// - Returns: the configured `Fluent` service, ready to be added to the service group.
public func callAsFunction() -> Fluent {
let fluent = Fluent(
@@ -63,8 +61,8 @@ public struct Service: Sendable {
isDefault: true
)
case .inMemory:
// A single connection keeps every query pointed at the same in-memory store,
// rather than each pooled connection getting its own private database.
// A single connection keeps every query pointed at the same in-memory store, rather than each pooled
// connection getting its own private database.
fluent.databases.use(
.sqlite(.memory, maxConnectionsPerEventLoop: 1),
as: .sqlite,
@@ -1,7 +1,6 @@
/// The connection parameters for the MySQL/MariaDB backend.
///
/// The executable builds this from its `database.*` configuration; the package itself reads no
/// configuration, so these values arrive as plain data.
/// The executable builds this from its `database.*` configuration; the package itself reads no configuration, so these values arrive as plain data.
public struct Configuration: Sendable {
// MARK: Properties
@@ -1,9 +1,16 @@
import Logging
import MySQLNIO
import NIOCore
import NIOPosix
import NIOSSL
import Testing
@testable import Persistence
@Suite("TLS enumeration")
@Suite(
"TLS enumeration",
.tags(.enumeration)
)
struct TLSTests {
// MARK: Properties tests
@@ -25,4 +32,27 @@ struct TLSTests {
#expect(configuration.bestEffortEquals(.makeClientConfiguration()))
}
@Test
func `prefer falls back to plaintext when the server offers no TLS`() async throws {
// The fake server never advertises `CLIENT_SSL`, so this connection can only succeed by downgrading to
// plaintext pinning the driver behavior the `prefer` posture relies on.
let server = try await PlaintextMySQLServer.start()
let tlsConfiguration = try #require(TLS.prefer.tlsConfiguration)
let connection = try await MySQLConnection.connect(
to: .init(ipAddress: "127.0.0.1", port: server.port),
username: "loud",
database: "loud",
tlsConfiguration: tlsConfiguration,
logger: Logger(label: "test"),
on: MultiThreadedEventLoopGroup.singleton.any()
).get()
let isConnected = !connection.isClosed
try await connection.close().get()
try await server.stop()
#expect(isConnected)
}
}
@@ -5,7 +5,10 @@ import Testing
@testable import Persistence
@Suite("Probe method")
@Suite(
"Probe method",
.tags(.method)
)
struct ProbeTests {
// MARK: Methods tests
@@ -5,7 +5,10 @@ import Testing
@testable import Persistence
@Suite("Service method")
@Suite(
"Service method",
.tags(.method)
)
struct ServiceTests {
// MARK: Methods tests
@@ -0,0 +1,8 @@
import Testing
extension Tag {
/// Tests exercising an enumeration of the Persistence package.
@Tag static var enumeration: Tag
/// Tests exercising a method of the Persistence package.
@Tag static var method: Tag
}
@@ -0,0 +1,162 @@
import NIOCore
import NIOPosix
/// A fake MySQL server speaking just enough of the wire protocol to complete a plaintext handshake.
///
/// Its greeting advertises the `mysql_native_password` plugin but **not** the `CLIENT_SSL` capability, and
/// it answers the client's handshake response with a bare OK packet so a client asking for TLS can only
/// end up connected in plaintext. This is what the `prefer` fallback test connects to, proving the driver
/// downgrades to plaintext rather than refusing the connection.
final class PlaintextMySQLServer {
// MARK: Properties
/// The port the server listens on, assigned by the system at bind time.
let port: Int
/// The listening channel the server accepts connections through.
private let channel: Channel
// MARK: Initializers
private init(
channel: Channel,
port: Int
) {
self.channel = channel
self.port = port
}
// MARK: Functions
/// Starts a server on the loopback interface, on a system-assigned port.
/// - Returns: the running server, ready to be connected to at ``port``.
static func start() async throws -> PlaintextMySQLServer {
let channel = try await ServerBootstrap(group: MultiThreadedEventLoopGroup.singleton)
.childChannelInitializer { channel in
channel.eventLoop.makeCompletedFuture {
try channel.pipeline.syncOperations.addHandler(Handler())
}
}
.bind(host: "127.0.0.1", port: 0)
.get()
guard let port = channel.localAddress?.port else {
throw ChannelError.unknownLocalAddress
}
return .init(
channel: channel,
port: port
)
}
/// Stops the server, closing its listening channel.
func stop() async throws {
try await channel.close().get()
}
}
// MARK: - Handlers
private extension PlaintextMySQLServer {
/// Greets a freshly accepted connection, accepts whatever authentication response arrives,
/// and closes on anything after that (e.g. a `COM_QUIT`).
final class Handler: ChannelInboundHandler {
// MARK: Type aliases
typealias InboundIn = ByteBuffer
typealias OutboundOut = ByteBuffer
// MARK: Properties
/// Whether the client's handshake response has already been answered with an OK packet.
private var didAuthenticate = false
// MARK: Functions
func channelActive(context: ChannelHandlerContext) {
context.writeAndFlush(
wrapOutboundOut(Self.greeting(allocator: context.channel.allocator)),
promise: nil
)
}
func channelRead(
context: ChannelHandlerContext,
data: NIOAny
) {
guard didAuthenticate else {
didAuthenticate = true
context.writeAndFlush(
wrapOutboundOut(Self.ok(allocator: context.channel.allocator)),
promise: nil
)
return
}
context.close(promise: nil)
}
// MARK: Helpers
/// The `HandshakeV10` greeting, framed and ready to send as the connection's first packet.
///
/// The advertised capabilities are `CLIENT_LONG_PASSWORD`, `CLIENT_PROTOCOL_41`,
/// `CLIENT_SECURE_CONNECTION`, and `CLIENT_PLUGIN_AUTH` deliberately **not** `CLIENT_SSL`,
/// so the client cannot upgrade the connection to TLS.
private static func greeting(allocator: ByteBufferAllocator) -> ByteBuffer {
var payload = allocator.buffer(capacity: 80)
payload.writeInteger(10, endianness: .little, as: UInt8.self) // protocol version
payload.writeNullTerminatedString("8.0.0") // server version
payload.writeInteger(1, endianness: .little, as: UInt32.self) // connection id
payload.writeBytes([1, 2, 3, 4, 5, 6, 7, 8]) // auth plugin data, part 1
payload.writeInteger(0, endianness: .little, as: UInt8.self) // filler
payload.writeInteger(0x8201, endianness: .little, as: UInt16.self) // capabilities, lower: LONG_PASSWORD | PROTOCOL_41 | SECURE_CONNECTION
payload.writeInteger(0x21, endianness: .little, as: UInt8.self) // character set (utf8)
payload.writeInteger(0x0002, endianness: .little, as: UInt16.self) // status flags (autocommit)
payload.writeInteger(0x0008, endianness: .little, as: UInt16.self) // capabilities, upper: PLUGIN_AUTH
payload.writeInteger(21, endianness: .little, as: UInt8.self) // auth plugin data length
payload.writeBytes([UInt8](repeating: 0, count: 10)) // reserved
payload.writeBytes([9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 0]) // auth plugin data, part 2
payload.writeNullTerminatedString("mysql_native_password") // auth plugin name
return framed(payload, sequence: 0, allocator: allocator)
}
/// A bare OK packet, framed as the reply to the client's handshake response.
private static func ok(allocator: ByteBufferAllocator) -> ByteBuffer {
var payload = allocator.buffer(capacity: 8)
payload.writeBytes([0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00]) // OK, no rows, autocommit, no warnings
return framed(payload, sequence: 2, allocator: allocator)
}
/// Wraps a payload in the MySQL packet frame: a 3-byte little-endian length and a sequence byte.
private static func framed(
_ payload: ByteBuffer,
sequence: UInt8,
allocator: ByteBufferAllocator
) -> ByteBuffer {
var packet = allocator.buffer(capacity: payload.readableBytes + 4)
var payload = payload
packet.writeInteger(UInt8(payload.readableBytes & 0xff))
packet.writeInteger(UInt8((payload.readableBytes >> 8) & 0xff))
packet.writeInteger(UInt8((payload.readableBytes >> 16) & 0xff))
packet.writeInteger(sequence)
packet.writeBuffer(&payload)
return packet
}
}
}