msflib-drivelink¶

Purpose¶

msflib-drivelink is a virtual filesystem, in the style of a cloud drive, for your users' files. It stores a tree of folders and files as database rows (DrivelinkNode), keeps the file bytes in whichever storage backend your core settings select, and exposes a REST API for browsing, uploading, renaming, moving, copying, trashing, restoring and searching. Each user sees their own tree; a workspace-scoped tree is available when you supply a workspace dependency.

It is a storage and organization layer only. It does not read file contents, and indexing for search or RAG lives in documents, which can follow drivelink changes through the event bus.

This package has no README. This page is written from the source and tests.

Install¶

[tool.poetry.dependencies]
msflib = { git = "https://github.com/msflib/fastapi.git", subdirectory = "core", rev = "core-v0.2.1" }
msflib-account = { git = "https://github.com/msflib/fastapi.git", subdirectory = "modules/account", rev = "account-v0.2.2" }
msflib-drivelink = { git = "https://github.com/msflib/fastapi.git", subdirectory = "modules/drivelink", rev = "drivelink-v0.2.0" }

It also depends on python-multipart for file uploads. There are no extras. The drivelinknode table has a foreign key to account.id.

Wiring into a host app¶

Add DrivelinkSettings to your settings class and mount the router. The router factory is router in msflib.drivelink.router. Import msflib.drivelink.models so the drivelinknode table is registered before you create tables.

from fastapi import FastAPI
from msflib.account.config import AccountSettings
from msflib.core.config import CoreSettings, SettingsBase
from msflib.drivelink.config import DrivelinkSettings
import msflib.drivelink.models  # registers the drivelinknode table
from msflib.drivelink.router import router as drivelink_router


class AppSettings(DrivelinkSettings, AccountSettings, CoreSettings, SettingsBase):
    pass


settings = AppSettings()
app = FastAPI()
app.include_router(
    drivelink_router(
        get_session=get_session,
        get_current_account=get_current_account,
        settings=settings,
        get_current_workspace=get_current_workspace,  # optional
        prefix="/drivelink",
    )
)

get_current_workspace is optional. When omitted, every request works on the account's personal tree (workspace_id is NULL). When given, it must return an object with an integer id (the WorkspaceContract shape from msflib.account.contracts), and the request works on that workspace's tree. A workspace tree and a personal tree never mix: all lookups filter on the exact workspace_id, and NULL is matched literally, not treated as a wildcard.

Routes, all relative to the prefix:

Method and path	Effect
`GET /health`	`{"module", "enabled", "prefix"}`
`GET /nodes`	List children of `parent_id` (omit for root). Query: `drivelink_id`, `trash` (`exclude`, `only`, `include`), `offset`, `limit` (max 500), `sort` (`name`, `path`, `created_at`, `updated_at`, `size_bytes`, `kind`), `order`
`GET /nodes/search?q=`	Case-insensitive substring search on name or path, excluding trashed nodes (`limit` max 200)
`GET /nodes/{node_id}`	One node
`POST /nodes/folders`	Create a folder; body `{"parent_id", "name", "drivelink_id"}`
`POST /nodes/{parent_id}/files`	Upload a file into a folder (multipart `file`, optional `name`)
`PUT /nodes/{node_id}/content`	Replace a file's bytes; bumps `version`
`PATCH /nodes/{node_id}`	Rename, move (`parent_id`), star, `color_label`, `custom_properties`
`POST /nodes/{node_id}/copy`	Copy to `{"parent_id", "name"}` (folders copy recursively)
`DELETE /nodes/{node_id}`	Move to trash; `?permanent=true` deletes for good
`POST /nodes/{node_id}/restore`	Restore from trash
`GET /trash`	Trashed children of `parent_id` (root if omitted)
`GET /nodes/{node_id}/download`	Signed URL: redirects by default, or `{"url", "expires_in"}` with `?redirect=false` (`expires_in` 60 to 86400)
`GET /recents`, `GET /starred`	Recently modified files; starred nodes
`POST /batch`	Apply `move`, `delete`, `rename` and `copy` operations atomically

Every route except GET /health requires an authenticated account and only returns nodes owned by it. GET /health depends only on get_session, so it is safe to use as a liveness check. There is no per-node sharing: ownership is account_id, and the can_share capability in responses is a flag derived from SHARING_ENABLED, not a sharing implementation.

Files cannot be uploaded to the root: POST /nodes/{parent_id}/files takes a required folder id, so create a folder first.

Known issue (#275)

Two gaps in the current behaviour: there is no way to upload a file at the root (create a folder first), and trashed nodes are never purged. The module only sets trash_expires_at, so run your own cleanup (see the Trash notes under Key concepts and Troubleshooting).

Configuration¶

DrivelinkSettings has the DRIVELINK namespace.

Key	Default	Effect
`ENABLED`	`true`	Only reported as `enabled` by `GET /health`; it does not switch any route off (#278)
`MAX_UPLOAD_BYTES`	50 MiB	Upload size limit, checked on upload and replace (HTTP 413)
`QUOTA_MAX_BYTES`	`None`	Per-account storage cap across all of the account's nodes. `None` means unlimited. Exceeding it raises "Storage quota exceeded." (HTTP 413)
`ALLOWED_MIME_PREFIXES`	`None`	If set, uploads whose content type does not start with one of the prefixes get HTTP 415
`TRASH_RETENTION_DAYS`	`None`	Days recorded in `trash_expires_at` when a node is trashed. `None` or a negative value means 30
`MAX_PATH_DEPTH`	`64`	Maximum folder depth when creating folders
`MAX_CHILDREN_PER_FOLDER`	`10000`	Cap on active children per folder
`STORAGE_PARENT_FOLDER`	`drivelink`	Parent folder or key prefix used in storage
`STORAGE_BUCKET_PREFIX`	`files`	Per-account storage bucket is `{prefix}-{account_id}`
`SHARING_ENABLED`	`false`	Only sets `capabilities.can_share` in responses; no sharing logic exists behind it (#278)
`MULTIPART_UPLOAD_ENABLED`	`false`	Declared, currently has no effect (#278). Defined but not used anywhere in this package

Every key has a flat alias. With subclassed settings (as above) use the flat environment variable, for example MAX_UPLOAD_BYTES=10485760. DRIVELINK__MAX_UPLOAD_BYTES only works if you declare the namespace as a field (DRIVELINK: DrivelinkSettings = DrivelinkSettings()). Both were checked. Note that the flat name ENABLED is shared by every module that defines it. With several modules composed as nested fields, DRIVELINK__ENABLED is unreliable because of that shared alias (in a host with documents and drivelink, DOCUMENTS__ENABLED=false was ignored while DRIVELINK__ENABLED=false switched off both); set the field in code instead, for example DrivelinkSettings(ENABLED=False).

Behaviour may change (#280)

The NAMESPACE__KEY environment variable style only works for settings composed as a field, not for subclassed hosts. This may change.

Where bytes are stored is decided by core, not by drivelink: CORE.STORAGE_METHOD (file, cloudinary, s3, gcs or azure) and, for local storage, CORE.STORAGE_PATH and CORE.STORAGE_BASE_URL. See core.

Key concepts¶

DrivelinkNode: one row per folder or file. Key fields: kind (folder or file), name, path (a slash-separated string rebuilt on rename and move), parent_id, account_id, workspace_id, drivelink_id, the storage_* columns, mime_type, size_bytes, version, deleted_at, trash_expires_at, is_starred, color_label and custom_properties.
Scope keys: account_id (owner), workspace_id (nullable, no foreign key) and drivelink_id (an optional integer that partitions several independent roots inside one account or workspace). The request routes pass drivelink_id as a query parameter. There is no tenant column on the node.

Behaviour may change (#267)

Scope profiles currently require tenant_id but allow a null workspace_id, so the two dimensions are not treated the same, and the drivelink node has a workspace_id and no tenant_id. Whether this should change is undecided.

DrivelinkNodeRead: the response model. It adds is_trashed, thumbnail_url (the storage URL for image files) and capabilities (can_rename, can_delete, can_download, can_share).
DrivelinkAction(settings) (msflib.drivelink.actions): the domain logic. Methods include create_folder, list_children, get_node_for_owner, rename_node, move_node, copy_node, soft_delete, restore, hard_delete, search_nodes, recent_files, starred_nodes, apply_batch, create_file_from_blob, replace_file_content and total_bytes_used. Scope arguments are keyword-only; domain errors are raised as ValueError with a message the router maps to HTTP status codes.
DrivelinkService(settings) (msflib.drivelink.services): the upload layer. It validates an UploadFile against the size and MIME settings, writes it to storage and then calls the action.
Names: names must be non-empty after trimming and may not be . or .. or contain / or NUL. A sibling with the same name in the same parent returns HTTP 409 ("already exists"). The model docstring recommends a partial unique index on (parent_id, name) WHERE deleted_at IS NULL in your migrations; the module does not create one.
Trash: DELETE sets deleted_at and trash_expires_at on the node and all of its descendants. Restore needs the parent to be live and name-collision free. Hard delete removes rows and also deletes the blob unless another node (including a trashed one) still refers to the same storage object. Nothing in this package purges expired trash on its own; trash_expires_at is a timestamp for your own cleanup job.
Copying files creates a new node that shares the original's blob rather than duplicating bytes; the blob is deleted only when its last referencing node is hard-deleted. Replacing a file's content writes a new blob, increments version, and removes the old blob if nothing else refers to it.
Error mapping in the router: messages containing "not found" give 404; "already exists" or "collision" give 409; "quota exceeded" or "too large" give 413; everything else raised as ValueError gives 400.

Events¶

DrivelinkAction is a ModelAction, so it emits the standard lifecycle events on the core event bus: drivelinknode-create-pre-commit, drivelinknode-update-pre-commit and drivelinknode-delete-pre-commit. Listeners receive (node, {"session": session, ...}). The documents module uses these to ingest files; see documents.

Register your own listeners on the app emitter:

from msflib.eventbus import bind_app_emitter

app_emitter = bind_app_emitter(app)


@app_emitter.on("drivelinknode-create-pre-commit")
def on_node_created(node, options):
    print("created", node.path)

Events reach app-bound listeners only while the app emitter is active: inside a request (including FastAPI BackgroundTasks), or in your own scripts and threads wrapped in with use_app_emitter(app): (from msflib.eventbus). Outside that, the default emitter is used and app listeners, including the documents drivelink hooks, are not called. Calling DrivelinkAction(...).create_folder(session, ...) from a script fires nothing on app_emitter unless it runs inside that with block, and a worker process, which has no app, needs its listeners on AppEmitter(get_emitter()). msflib.drivelink.documents_contract documents the identity convention (drivelink_id, node_id, version) for external consumers. Note that msflib.drivelink exports only documents_contract from its package __init__; import everything else from its submodules.

Examples¶

Create a folder, upload a file, and manage it through the router. This ran against an in-memory SQLite database and local file storage:

import tempfile
from fastapi import FastAPI
from fastapi.testclient import TestClient
from msflib.account.config import AccountSettings
from msflib.core.config import CoreSettings, SettingsBase
from msflib.drivelink.config import DrivelinkSettings
import msflib.drivelink.models
from msflib.drivelink.router import router

class AppSettings(DrivelinkSettings, AccountSettings, CoreSettings, SettingsBase):
    pass

settings = AppSettings(STORAGE_PATH=tempfile.mkdtemp(), QUOTA_MAX_BYTES=1_000_000)
# ... create engine, tables and an `account`, then:
app = FastAPI()
app.include_router(router(get_session=lambda: session, get_current_account=lambda: account,
                          settings=settings, prefix="/drivelink"))
c = TestClient(app)

folder = c.post("/drivelink/nodes/folders", json={"parent_id": None, "name": "Reports"}).json()
# folder["path"] == "/Reports", folder["kind"] == "folder"

up = c.post(f"/drivelink/nodes/{folder['id']}/files",
            files={"file": ("q1.txt", b"hello", "text/plain")})
node = up.json()   # 200, name "q1.txt", size_bytes 5, capabilities.can_download True

c.get(f"/drivelink/nodes/{node['id']}/download?redirect=false").json()
# {"url": "/uploads/drivelink/files-1/<uuid>-q1.txt", "expires_in": 3600}

c.post("/drivelink/nodes/folders", json={"parent_id": None, "name": "Reports"}).status_code  # 409
c.delete(f"/drivelink/nodes/{node['id']}").json()   # {"deleted": True, "permanent": False}
c.get(f"/drivelink/trash?parent_id={folder['id']}").json()   # the trashed q1.txt
c.post(f"/drivelink/nodes/{node['id']}/restore")    # 200

Run several changes atomically with /batch. Each operation has an id you choose, an op (move, delete, rename or copy) and its arguments. Any failure rolls the whole batch back:

c.post("/drivelink/batch", json={"ops": [
    {"id": "1", "op": "rename", "node_id": node["id"], "name": "q1-final.txt"},
]}).json()
# {"results": [{"id": "1", "status": "ok", "node_id": 2}]}

Use the action directly for server-side code. These calls are keyword-only after the session:

from msflib.drivelink.actions import DrivelinkAction

action = DrivelinkAction(settings=settings)
folder = action.create_folder(session, account_id=account.id, parent_id=None, name="Inbox")
children = action.list_children(session, account_id=account.id, parent_id=folder.id)

This last snippet was run against the same database as the router example. Called like this, outside a request, it does not fire app-bound event listeners; see Events.

Troubleshooting¶

404 Node not found. for a node you just created. The node is looked up with the same workspace_id and drivelink_id as the request. A node created inside a workspace is invisible to requests without a workspace, and the reverse. Pass the same drivelink_id query parameter you created it under.
422 on upload. Send a multipart request with the file in a form field named file.
413 on upload. The file exceeds MAX_UPLOAD_BYTES, or the account would exceed QUOTA_MAX_BYTES.
415 on upload. ALLOWED_MIME_PREFIXES is set and the content type does not start with one of the prefixes.
409 when restoring. A live node with the same name now exists in the destination. Rename or delete it first. "Restore parent folder first." means the parent is still in trash.
Trashed items never disappear. The module only sets trash_expires_at; it ships no purge job. Delete expired nodes with DrivelinkAction.hard_delete, or DELETE ...?permanent=true, from your own scheduler.
Download gives a relative URL like /uploads/.... With local file storage the "signed" URL is the stored URL under CORE.STORAGE_BASE_URL; your host app must serve that path (for example with a static files mount). If a backend raises HTTP 501 for signing, the router falls back to the raw storage_url.
DRIVELINK__... environment variables are ignored. See Configuration: with subclassed settings use the flat names.
Table is missing at startup. Import msflib.drivelink.models before creating tables, since the package's __init__ does not import the models.

API reference¶

See the generated API reference for msflib.drivelink. The package has no README.