diff --git a/Documentation/ExampleFiles.png b/Documentation/ExampleFiles.png new file mode 100644 index 0000000..18966a1 Binary files /dev/null and b/Documentation/ExampleFiles.png differ diff --git a/Documentation/Plan.md b/Documentation/Plan.md index f355360..b9d7727 100644 --- a/Documentation/Plan.md +++ b/Documentation/Plan.md @@ -46,7 +46,7 @@ AppRuntime's contracts. | arm64 and x86_64 Linux with glibc (Debian 13, Ubuntu 24.04+) | armv7, musl | | Native slices linked into the host | wasm2c (AppRuntime's shim, as on Darwin); WasmKit for development | | Wayland and X11 through SDL3; a headless driver for tests | Multiple windows | -| `Renderer`, Bluetooth `Central` and friends, GL surfaces, web views, the built-ins | Location (GeoClue), Device (UPower, sysfs), Networking (libcurl), Formatting (ICU), Motion (iio-sensor-proxy), NFC (neard) | +| `Renderer`, Bluetooth `Central` and friends, GL surfaces, web views, files, the built-ins | Location (GeoClue), Device (UPower, sysfs), Networking (libcurl), Formatting (ICU), Motion (iio-sensor-proxy), NFC (neard) | ## 2. Process model @@ -87,13 +87,13 @@ timers for `schedule`. | Module | Contents | |---|---| -| `LinuxHost` | `AppHost` (built-ins, modules, `rt_main`), `MainLoop`, `MainLoopClock`, the `Bundle` built-in; over AppRuntime's `RuntimeHost`, which every host shares: `AppInfo` (the bundle's `Info.plist` keys), the compatibility checks, `HostModule`, and the `Permissions` and `Application` built-ins | +| `LinuxHost` | `AppHost` (built-ins, modules, `rt_main`), `MainLoop`, `MainLoopClock`, the `Bundle` built-in, and `LinuxFiles` (§9); over AppRuntime's `RuntimeHost`, which every host shares: `AppInfo` (the bundle's `Info.plist` keys), the compatibility checks, `HostModule`, the `Permissions` and `Application` built-ins, and `HostFiles` | | `LinuxUI` | The SwiftUI player: DarwinAppRuntime's `DarwinUI`, copied by `Scripts/sync-darwinui.sh`, over Cacao's `SwiftUI` and `Combine` modules; `Host.swift` gives it the logger and bundle it takes from its host | | `LinuxBluetooth` | `Central`, `Peripheral`, `Service`, `Characteristic`, `Descriptor`, `Advertisement` over BlueZ | | `LinuxGraphics` | `Surface` over EGL and GLES 2.0, loaded at run time; the player's `GLSurface` view shows it (§7) | | `LinuxWebKit` | `WebContent` over WPE WebKit, loaded at run time; the player's `WebView` view shows it (§8) | | `LinuxHostApp` | `main`: reads the app's keys, starts the host, and runs Cacao's SDL loop with one window showing the guest's first renderer | -| `LinuxPackager`, `linux-app` | The packager (§10, [Packager.md](Packager.md)) | +| `LinuxPackager`, `linux-app` | The packager (§11, [Packager.md](Packager.md)) | The package depends on AppRuntime without its `GuestExports` trait, like DarwinAppRuntime, so the slice's `rt_dispatch` and `rt_finalize` are the @@ -275,13 +275,41 @@ WebKit, and the player draws `WebView` nodes. It follows Perry's - Reading files beside a bundled page only: WebKit's file-URL rules apply. -## 9. Testing +## 9. Files + +`LinuxFiles` is the `Files` contract (AppRuntime's `Files.md`): AppRuntime's +`HostFiles`, which does it all over POSIX calls, with the app's directories +where Linux keeps them, and answers posted to the main loop from its queue. + +| Directory | In a container | Otherwise, XDG's | +|---|---|---| +| `documents` | `/Documents` | `$XDG_DATA_HOME//Documents` | +| `data` | `/Data` | `$XDG_DATA_HOME//Data` | +| `caches` | `/Caches` | `$XDG_CACHE_HOME/` | +| `temporary` | `/tmp` | `$XDG_RUNTIME_DIR/`, or `/tmp/-` | +| `bundle` | the bundle's `resources/` | the same | + +- **``** is the bundle identifier, which must name a directory. + `XDG_DATA_HOME` is `~/.local/share` and `XDG_CACHE_HOME` `~/.cache` when + they're unset or relative, as the XDG specification says. +- **A container** is `LINUXAPPRUNTIME_CONTAINER`, an absolute path: + - a PortMaster port's is its `conf/` folder, where ports keep what's + theirs; + - a launcher's sandbox would pass `/data` (AppRuntime's §7.8). +- **`temporary` is the app's own**, never `/tmp` itself, since `HostFiles` + empties it at every start. +- **If the directories can't be decided**, for example with no `HOME`, the + host logs why and every operation fails through its callback. + +## 10. Testing - `LinuxUI`: DarwinUI's tests, on Cacao (§5). - `LinuxBluetooth`: the contract against a fake `BlueZBus` that plays BlueZ's objects and signals. Runs on macOS and Linux. - `LinuxGraphics`: a surface's input order, focus and size, and on Linux frames drawn and read back with Mesa's llvmpipe, and a lost context. +- `LinuxHost`'s files: the directories from the environment, and the + contract end to end in a container, answering on the main loop. - `LinuxWebKit`: on Linux, in WPE: loading HTML and the events' order, scripts' JSON, navigations asked about and refused, back, bundle resources, a refused connection, remote loads without `network`, frames @@ -293,7 +321,7 @@ WebKit, and the player draws `WebView` nodes. It follows Perry's - Real BlueZ needs hardware: manual runs on a Raspberry Pi or a laptop with `bluetoothd`. -## 10. Phases +## 11. Phases 1. **Host runtime and player**: `LinuxHost`, `LinuxUI`, `LinuxBluetooth`, `LinuxHostApp`; `ExampleUI` runs in Docker. @@ -308,7 +336,7 @@ WebKit, and the player draws `WebView` nodes. It follows Perry's 4. **Player**: what DarwinUI and Cacao add: Cacao's AT-SPI bridge, input replay for end-to-end tests. -## 11. Open questions +## 12. Open questions - **Permissions.** Desktop Linux has no prompts for Bluetooth; Flatpak's portals cover location and the camera. `Permissions.request` answers from diff --git a/Documentation/PortMaster.md b/Documentation/PortMaster.md index 3667769..d0fc465 100644 --- a/Documentation/PortMaster.md +++ b/Documentation/PortMaster.md @@ -50,6 +50,7 @@ host's options for a handheld: | `LINUXAPPRUNTIME_SCALE` | Pixels per point: 1 below 700 pixels tall, 1.5 below 1000, 2 above, so a 640 × 480 screen is 640 × 480 points and a 720 × 720 one 480 × 480 | | `LINUXAPPRUNTIME_CURSOR=1` | Draw a pointer: there's no mouse, and KMSDRM has no system cursor | | `LINUXAPPRUNTIME_FONTS` | `fonts/`, used when fontconfig isn't there | +| `LINUXAPPRUNTIME_CONTAINER` | `conf/`: the app's files, where ports keep what's theirs | ## 3. Controls diff --git a/Package.swift b/Package.swift index 6c88ced..0d23048 100644 --- a/Package.swift +++ b/Package.swift @@ -92,7 +92,8 @@ let package = Package( .target( name: "LinuxHost", - dependencies: [runtime("CRuntime"), runtime("RuntimeCore"), runtime("RuntimeNative"), runtime("DynamicRuntime"), runtime("RuntimeABI"), runtime("RuntimeHost")], + dependencies: [runtime("CRuntime"), runtime("RuntimeCore"), runtime("RuntimeNative"), runtime("DynamicRuntime"), runtime("RuntimeABI"), + runtime("RuntimeHost"), runtime("DynamicFiles")], swiftSettings: host ), @@ -175,7 +176,7 @@ let package = Package( // MARK: - Tests - .testTarget(name: "LinuxHostTests", dependencies: ["LinuxHost"], swiftSettings: host), + .testTarget(name: "LinuxHostTests", dependencies: ["LinuxHost", runtime("DynamicFiles")], swiftSettings: host), .testTarget( name: "LinuxUITests", dependencies: ["LinuxUI", runtime("DynamicUI"), runtime("DynamicRuntime"), .product(name: "Cacao", package: "Cacao")] + swiftUI, diff --git a/README.md b/README.md index 2cbc7aa..e3a98d8 100644 --- a/README.md +++ b/README.md @@ -36,11 +36,16 @@ llvmpipe: ![ExampleWeb on Linux](Documentation/ExampleWeb.png) +`ExampleFiles` reads its welcome text from the bundle and lists the notes +in its `documents`: + +![ExampleFiles on Linux](Documentation/ExampleFiles.png) + ## Modules | Module | | |---|---| -| `LinuxHost` | `AppHost` (built-ins, modules, `rt_main`), `MainLoop` and its clock, and the `Bundle` built-in, over AppRuntime's shared `RuntimeHost` (`AppInfo`, compatibility checks, `HostModule`, `Permissions`, `Application`) | +| `LinuxHost` | `AppHost` (built-ins, modules, `rt_main`), `MainLoop` and its clock, the `Bundle` built-in, and `LinuxFiles`, the app's files in XDG's directories or a container ([plan](Documentation/Plan.md) §9), over AppRuntime's shared `RuntimeHost` (`AppInfo`, compatibility checks, `HostModule`, `Permissions`, `Application`, `HostFiles`) | | `LinuxUI` | The SwiftUI player: DarwinAppRuntime's `DarwinUI`, copied by `Scripts/sync-darwinui.sh`, over Cacao's `SwiftUI` | | `LinuxBluetooth` | `Central`, `Peripheral`, `Service`, `Characteristic`, `Descriptor` and `Advertisement` over BlueZ, through `wendylabsinc/dbus` | | `LinuxGraphics` | `Surface`: GLES 2.0 drawn in EGL pbuffers, loaded at run time, and read back into the player's `GLSurface` view ([plan](Documentation/Plan.md) §7) | @@ -144,6 +149,7 @@ The bundle needs `linux-arm64` and `linux-armv7` slices (`guest.py bundle | Variable | | |---|---| | `LINUXAPPRUNTIME_DEBUG` | `1` logs the host's debug messages | +| `LINUXAPPRUNTIME_CONTAINER` | An absolute path that holds the app's files (`Documents`, `Data`, `Caches`, `tmp`), in place of XDG's directories | | `CACAO_SCREENSHOT` | Saves the first frame to a PNG and quits | | `CACAO_SHOW_STATISTICS` | `1` shows Cacao's frame times | | `SDL_VIDEO_DRIVER` | `wayland`, `x11`, `kmsdrm`, or `offscreen` for no display | diff --git a/Sources/LinuxHost/LinuxFiles.swift b/Sources/LinuxHost/LinuxFiles.swift new file mode 100644 index 0000000..8ea186e --- /dev/null +++ b/Sources/LinuxHost/LinuxFiles.swift @@ -0,0 +1,115 @@ +import DynamicFiles +import DynamicRuntime +import Foundation +import RuntimeABI +import RuntimeHost + +/// The `Files` contract (AppRuntime's `Files.md`): `RuntimeHost`'s +/// `HostFiles`, with the app's directories where Linux keeps them, and +/// answers on the main loop (`Plan.md` §9). +/// +/// | Directory | `LINUXAPPRUNTIME_CONTAINER` set | Otherwise (XDG) | +/// |---|---|---| +/// | `documents` | `/Documents` | `$XDG_DATA_HOME//Documents` | +/// | `data` | `/Data` | `$XDG_DATA_HOME//Data` | +/// | `caches` | `/Caches` | `$XDG_CACHE_HOME/` | +/// | `temporary` | `/tmp` | `$XDG_RUNTIME_DIR/`, or `/tmp/-` | +/// | `bundle` | the bundle's `resources/` | the same | +/// +/// `` is the app's bundle identifier. `XDG_DATA_HOME` defaults to +/// `~/.local/share` and `XDG_CACHE_HOME` to `~/.cache`. `temporary` is the +/// app's own, since it's emptied at every start. +public enum LinuxFiles: HostModule { + + public static var contracts: [ContractDescriptor] { + HostFilesModule.contracts + } + + /// Says where the app's files are, then registers `Files`. The + /// directories come from the app's identifier and `LinuxBundle`, which + /// `AppHost` sets up first. + public static func register(in registrar: some ContractRegistrar) { + let identifier = LinuxBundle.info.identifier + do { + let directories = try self.directories(identifier: identifier, resources: LinuxBundle.resources, environment: environment) + try HostFiles.configure(HostFiles.Configuration(directories: directories, deliver: deliver)) + HostLog.debug("files: \(directories.sorted { $0.key.rawValue < $1.key.rawValue }.map { "\($0.key): \($0.value)" }.joined(separator: ", "))") + } catch { + // Every operation then fails, and says so through its callback. + HostLog.error("files: \(error)") + } + HostFilesModule.register(in: registrar) + } + + /// The environment the directories come from; replaceable for tests. + nonisolated(unsafe) public static var environment = ProcessInfo.processInfo.environment + + /// Answers go to the main loop, from `HostFiles`' queue. + static let deliver: @Sendable (@escaping @Sendable () -> Void) -> Void = { answer in + MainLoop.shared.post(answer) + } + + /// Where each directory is for the app `identifier`. + public static func directories( + identifier: String, + resources: String?, + environment: [String: String], + userID: UInt32 = getuid() + ) throws(LinuxFilesError) -> [FileDirectory: String] { + guard isPathComponent(identifier) else { + throw LinuxFilesError("the bundle identifier \"\(identifier)\" can't name a directory") + } + var directories: [FileDirectory: String] = [:] + if let container = nonEmpty(environment["LINUXAPPRUNTIME_CONTAINER"]) { + guard container.hasPrefix("/") else { + throw LinuxFilesError("LINUXAPPRUNTIME_CONTAINER must be an absolute path") + } + directories[.documents] = container + "/Documents" + directories[.data] = container + "/Data" + directories[.caches] = container + "/Caches" + directories[.temporary] = container + "/tmp" + } else { + guard let home = nonEmpty(environment["HOME"]) else { + throw LinuxFilesError("neither LINUXAPPRUNTIME_CONTAINER nor HOME is set") + } + let data = absolute(environment["XDG_DATA_HOME"]) ?? home + "/.local/share" + let cache = absolute(environment["XDG_CACHE_HOME"]) ?? home + "/.cache" + directories[.documents] = data + "/" + identifier + "/Documents" + directories[.data] = data + "/" + identifier + "/Data" + directories[.caches] = cache + "/" + identifier + if let runtime = absolute(environment["XDG_RUNTIME_DIR"]) { + directories[.temporary] = runtime + "/" + identifier + } else { + directories[.temporary] = "/tmp/\(identifier)-\(userID)" + } + } + if let resources { + directories[.bundle] = resources + } + return directories + } + + private static func nonEmpty(_ value: String?) -> String? { + value.flatMap { $0.isEmpty ? nil : $0 } + } + + /// The XDG specification ignores relative paths in its variables. + private static func absolute(_ value: String?) -> String? { + nonEmpty(value).flatMap { $0.hasPrefix("/") ? $0 : nil } + } + + /// One path component: not empty, `.` or `..`, and no `/` or NUL. + static func isPathComponent(_ name: String) -> Bool { + !name.isEmpty && name != "." && name != ".." && !name.contains("/") && !name.contains("\0") + } +} + +/// Why the app's directories couldn't be decided. +public struct LinuxFilesError: Error, CustomStringConvertible { + + public var description: String + + init(_ description: String) { + self.description = description + } +} diff --git a/Sources/LinuxHostApp/HostApp.swift b/Sources/LinuxHostApp/HostApp.swift index fbd15c6..4cb4372 100644 --- a/Sources/LinuxHostApp/HostApp.swift +++ b/Sources/LinuxHostApp/HostApp.swift @@ -25,8 +25,11 @@ import LinuxUI public enum HostApp { /// Every module this host has: the SwiftUI player (DarwinUI, on Cacao), - /// Bluetooth, and GL surfaces and web pages, which the player shows. - public static let modules: [any HostModule.Type] = [DarwinUI.self, LinuxBluetooth.self, LinuxGraphics.self, LinuxWebKit.self] + /// Bluetooth, GL surfaces and web pages, which the player shows, and + /// the app's files. + public static let modules: [any HostModule.Type] = [ + DarwinUI.self, LinuxBluetooth.self, LinuxGraphics.self, LinuxWebKit.self, LinuxFiles.self, + ] /// The window's title: the app's name. nonisolated(unsafe) static var title = "" diff --git a/Sources/LinuxPackager/PortMaster.swift b/Sources/LinuxPackager/PortMaster.swift index dbd01b1..66895ec 100644 --- a/Sources/LinuxPackager/PortMaster.swift +++ b/Sources/LinuxPackager/PortMaster.swift @@ -241,8 +241,10 @@ public struct PortMasterPort { export TEXTINPUTINTERACTIVE="Y" # The host on a handheld: one fullscreen window, a drawn pointer, the - # port's fonts when fontconfig isn't there, and phone-sized layouts - # scaled to the screen. + # port's fonts when fontconfig isn't there, phone-sized layouts + # scaled to the screen, and the app's files in the port's conf/, + # where ports keep what's theirs. + export LINUXAPPRUNTIME_CONTAINER="$GAMEDIR/conf" export LINUXAPPRUNTIME_FULLSCREEN=1 export LINUXAPPRUNTIME_CURSOR=1 export LINUXAPPRUNTIME_FONTS="$GAMEDIR/fonts" diff --git a/Sources/LinuxPackager/Validator.swift b/Sources/LinuxPackager/Validator.swift index 2547090..89bcf11 100644 --- a/Sources/LinuxPackager/Validator.swift +++ b/Sources/LinuxPackager/Validator.swift @@ -12,9 +12,9 @@ public enum HostContracts { /// The modules `linux-app-host` registers (`HostApp.modules`): the /// player's contracts, without building the player, Bluetooth, - /// graphics and web views. + /// graphics, web views and files. nonisolated(unsafe) public static let modules: [any HostModule.Type] = [ - PlayerContracts.self, LinuxBluetooth.self, LinuxGraphics.self, LinuxWebKit.self, + PlayerContracts.self, LinuxBluetooth.self, LinuxGraphics.self, LinuxWebKit.self, LinuxFiles.self, ] public static var provided: [String: ContractDescriptor] { diff --git a/Tests/LinuxHostTests/LinuxFilesTests.swift b/Tests/LinuxHostTests/LinuxFilesTests.swift new file mode 100644 index 0000000..358a64b --- /dev/null +++ b/Tests/LinuxHostTests/LinuxFilesTests.swift @@ -0,0 +1,122 @@ +import DynamicFiles +import Foundation +import RuntimeHost +import Testing +@testable import LinuxHost + +/// Records the one answer an operation gets. +final class RecordingCallback: FileCallback { + + var answer: String? + + func didComplete() { answer = "complete" } + + func didRead(_ data: [UInt8]) { answer = "read \(String(decoding: data, as: UTF8.self))" } + + func didList(_ names: [String]) { answer = "list \(names.joined(separator: ","))" } + + func didGetAttributes(kind: Int32, size: Int64, modified: Int64) { + answer = "attributes \(kind) \(size) \(modified > 0)" + } + + func didFail(_ error: Int32) { answer = "fail \(error)" } +} + +/// `HostFiles` is configured once for the process: one test at a time. +@Suite(.serialized) struct LinuxFilesTests { + + @Test func xdgDirectories() throws { + let directories = try LinuxFiles.directories( + identifier: "org.example.App", resources: "/apps/App.app/resources", + environment: ["HOME": "/home/ada", "XDG_RUNTIME_DIR": "/run/user/1000"], userID: 1000) + #expect(directories == [ + .documents: "/home/ada/.local/share/org.example.App/Documents", + .data: "/home/ada/.local/share/org.example.App/Data", + .caches: "/home/ada/.cache/org.example.App", + .temporary: "/run/user/1000/org.example.App", + .bundle: "/apps/App.app/resources", + ]) + } + + @Test func xdgVariablesAndTheirDefaults() throws { + let directories = try LinuxFiles.directories( + identifier: "org.example.App", resources: nil, + environment: ["HOME": "/home/ada", "XDG_DATA_HOME": "/data/ada", "XDG_CACHE_HOME": "relative/is/ignored"], + userID: 1000) + #expect(directories[.documents] == "/data/ada/org.example.App/Documents") + #expect(directories[.caches] == "/home/ada/.cache/org.example.App") + // Without a runtime directory, the app's own under /tmp. + #expect(directories[.temporary] == "/tmp/org.example.App-1000") + // No bundle, no bundle directory: its files aren't found. + #expect(directories[.bundle] == nil) + } + + @Test func aContainer() throws { + let directories = try LinuxFiles.directories( + identifier: "org.example.App", resources: "/roms/ports/app/App.app/resources", + environment: ["HOME": "/home/ada", "LINUXAPPRUNTIME_CONTAINER": "/roms/ports/app/conf"]) + #expect(directories == [ + .documents: "/roms/ports/app/conf/Documents", + .data: "/roms/ports/app/conf/Data", + .caches: "/roms/ports/app/conf/Caches", + .temporary: "/roms/ports/app/conf/tmp", + .bundle: "/roms/ports/app/App.app/resources", + ]) + #expect(throws: LinuxFilesError.self) { + try LinuxFiles.directories(identifier: "org.example.App", resources: nil, + environment: ["LINUXAPPRUNTIME_CONTAINER": "conf"]) + } + } + + @Test func identifiersMustNameADirectory() { + for identifier in ["", ".", "..", "org/example", "a\0b"] { + #expect(throws: LinuxFilesError.self) { + try LinuxFiles.directories(identifier: identifier, resources: nil, environment: ["HOME": "/home/ada"]) + } + } + #expect(throws: LinuxFilesError.self) { + try LinuxFiles.directories(identifier: "org.example.App", resources: nil, environment: [:]) + } + } + + /// The contract end to end: `HostFiles` in a container, answering on + /// the main loop. + @Test func filesAnswerOnTheMainLoop() throws { + let root = FileManager.default.temporaryDirectory.appendingPathComponent("linux-files-\(UUID())") + let resources = root.appendingPathComponent("App.app/resources") + try FileManager.default.createDirectory(at: resources.appendingPathComponent("levels"), withIntermediateDirectories: true) + try Data("level one".utf8).write(to: resources.appendingPathComponent("levels/1.map")) + defer { try? FileManager.default.removeItem(at: root) } + let directories = try LinuxFiles.directories( + identifier: "org.example.App", resources: resources.path, + environment: ["LINUXAPPRUNTIME_CONTAINER": root.appendingPathComponent("conf").path]) + try HostFiles.configure(HostFiles.Configuration(directories: directories, deliver: LinuxFiles.deliver)) + let files = HostFiles.shared() + + func perform(_ operation: (RecordingCallback) -> Void) -> String? { + let callback = RecordingCallback() + operation(callback) + MainLoop.shared.run { callback.answer != nil } + return callback.answer + } + let documents = FileDirectory.documents.rawValue + // The writable directories are there from the start. + #expect(FileManager.default.fileExists(atPath: root.appendingPathComponent("conf/Documents").path)) + #expect(perform { files.read(FileDirectory.bundle.rawValue, path: "levels/1.map", offset: 0, length: -1, callback: $0) } + == "read level one") + #expect(perform { files.write(documents, path: "saves/slot1.sav", data: Array("saved".utf8), mode: 0, callback: $0) } + == "complete") + #expect(perform { files.write(documents, path: "saves/slot1.sav", data: Array(" more".utf8), mode: 1, callback: $0) } + == "complete") + #expect(perform { files.read(documents, path: "saves/slot1.sav", offset: 0, length: -1, callback: $0) } == "read saved more") + #expect(perform { files.list(documents, path: "", callback: $0) } == "list saves/") + #expect(perform { files.attributes(documents, path: "saves/slot1.sav", callback: $0) } == "attributes 0 10 true") + #expect(perform { files.copy(FileDirectory.bundle.rawValue, path: "levels", toDirectory: documents, toPath: "levels", callback: $0) } + == "complete") + #expect(perform { files.write(FileDirectory.bundle.rawValue, path: "x", data: [], mode: 0, callback: $0) } + == "fail \(FileError.readOnly.rawValue)") + #expect(perform { files.read(documents, path: "../escape", offset: 0, length: -1, callback: $0) } + == "fail \(FileError.invalidPath.rawValue)") + #expect(try String(contentsOf: root.appendingPathComponent("conf/Documents/levels/1.map"), encoding: .utf8) == "level one") + } +}