PHPackages padosoft/askmydocs-connector-base - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Authentication & Authorization](/categories/authentication) 4. / 5. padosoft/askmydocs-connector-base ActiveLibrary[Authentication & Authorization](/categories/authentication) padosoft/askmydocs-connector-base ================================= Framework primitives for AskMyDocs connectors — interface, base helpers, registry, OAuth vault, sync job, scheduler, exceptions. Extend ConnectorInterface to ingest any data source as RAG-ingestible knowledge. v1.4.0(1mo ago)11931Apache-2.0PHPPHP ^8.3CI passing Since May 12Pushed 1mo ago1 watchersCompare [ Source](https://github.com/padosoft/askmydocs-connector-base)[ Packagist](https://packagist.org/packages/padosoft/askmydocs-connector-base)[ RSS](/packages/padosoft-askmydocs-connector-base/feed)WikiDiscussions main Synced 2w ago READMEChangelog (7)Dependencies (24)Versions (8)Used By (1) askmydocs-connector-base ======================== [](#askmydocs-connector-base) **Framework primitives for AskMyDocs connectors — write a Laravel package, plug it into any AskMyDocs instance.** Implement `ConnectorInterface` on your favourite data source (Google Drive, Notion, Confluence, a CSV bucket, an internal API, ...) and let AskMyDocs ingest it as RAG-grounded knowledge with OAuth, encrypted-at-rest credentials, retry-aware queued syncs, per-tenant isolation, and a cadence scheduler — all wired automatically by composer discovery. [![CI status](https://camo.githubusercontent.com/fc43a5171890faf93a7fb82ae856adacecf2494bec47b44b11df308586fee855/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f616374696f6e732f776f726b666c6f772f7374617475732f7061646f736f66742f61736b6d79646f63732d636f6e6e6563746f722d626173652f74657374732e796d6c3f6272616e63683d6d61696e266c6162656c3d7465737473)](https://github.com/padosoft/askmydocs-connector-base/actions/workflows/tests.yml) [![Packagist version](https://camo.githubusercontent.com/58a23bf215adc8002c324a8b02c8d5b96d41838bec7cd1e9b0c3bfea5e07a5fe/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f7061646f736f66742f61736b6d79646f63732d636f6e6e6563746f722d626173652e7376673f6c6162656c3d7061636b6167697374)](https://packagist.org/packages/padosoft/askmydocs-connector-base) [![Total downloads](https://camo.githubusercontent.com/7f2090abe5e073cef7ca92bffd83c34bcff8e55ea03a3b4b67863246f59a1961/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f7061646f736f66742f61736b6d79646f63732d636f6e6e6563746f722d626173652e7376673f6c6162656c3d646f776e6c6f616473)](https://packagist.org/packages/padosoft/askmydocs-connector-base) [![License](https://camo.githubusercontent.com/798509b4df525f56802b56f8096862487f08023e3d7561c68656f8dab10d0d6e/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4170616368652d2d322e302d626c75652e737667)](LICENSE) [![PHP version](https://camo.githubusercontent.com/2457b6d694af1315d4c3b980954cd62f302ca8263328d648e57ac15df2f05974/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7068702d382e33253230253743253230382e34253230253743253230382e352d373737424234)](https://camo.githubusercontent.com/2457b6d694af1315d4c3b980954cd62f302ca8263328d648e57ac15df2f05974/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7068702d382e33253230253743253230382e34253230253743253230382e352d373737424234) [![Laravel version](https://camo.githubusercontent.com/26b345c07758850b08014603358538a5ede6d1aa67d23cc11fd9fdf9b45a704d/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c61726176656c2d313225323025374325323031332d464632443230)](https://camo.githubusercontent.com/26b345c07758850b08014603358538a5ede6d1aa67d23cc11fd9fdf9b45a704d/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c61726176656c2d313225323025374325323031332d464632443230) --- Table of contents ----------------- [](#table-of-contents) 1. [Why this package](#why-this-package) 2. [What you get](#what-you-get) 3. [Architecture at a glance](#architecture-at-a-glance) 4. [Installation](#installation) 5. [Quick start — write your first connector in 50 lines](#quick-start--write-your-first-connector-in-50-lines) 6. [The 10-method contract](#the-10-method-contract) 7. [Optional: credential form interface](#optional-credential-form-interface) 8. [How auto-discovery works](#how-auto-discovery-works) 9. [Credential vault — encrypted, atomic, tenant-scoped](#credential-vault--encrypted-atomic-tenant-scoped) 10. [Scheduler + sync job](#scheduler--sync-job) 11. [Multi-tenancy (R30 + R31)](#multi-tenancy-r30--r31) 12. [Configuration reference](#configuration-reference) 13. [Testing](#testing) 14. [Roadmap](#roadmap) 15. [License](#license) --- Why this package ---------------- [](#why-this-package) [AskMyDocs](https://github.com/lopadova/AskMyDocs) is an enterprise-grade RAG + canonical knowledge compilation system. Out of the box it ingests markdown from disk, the chat UI, an HTTP API, and a Git-driven workflow. But the knowledge people actually want to query lives in **Google Drive, Notion, Confluence, Jira, OneDrive, Evernote, Fabric, Slack, Salesforce, HubSpot, a private S3 bucket, a custom CRM** — anywhere except markdown-on-disk. This package is **the smallest possible surface** for shipping a new connector: - A 10-method `ConnectorInterface` you implement. - A `BaseConnector` that gives you OAuth state-token CSRF, credential refresh, and tenant-scoped installation lookup for free. - A registry that auto-discovers your package the moment somebody `composer require`s it — zero config edits on the consumer side. - An `OAuthCredentialVault` that handles encryption-at-rest, refresh-token semantics, and **atomic concurrent writes** (no read-modify-write data loss on shared cursor blobs). - A queued `ConnectorSyncJob` with exponential backoff, tenant restore, and failure-state recording. - A cadence scheduler that walks active installations every minute and dispatches due syncs. - Per-tenant isolation baked into every query (R30 / R31 — see [Multi-tenancy](#multi-tenancy-r30--r31)). > **Write the connector. Ship the package. Composer-require it from any AskMyDocs install. Done.** What you get ------------ [](#what-you-get) SurfaceClassWhat it doesContract`ConnectorInterface`10 methods every connector implementsBase`BaseConnector`OAuth state-token CSRF, refresh helper, tenant lookupRegistry`ConnectorRegistry`Boot-time R23 validation + composer-extra auto-discoveryVault`Auth\OAuthCredentialVault`AES-encrypted tokens, atomic `setExtraKey` (R21), tenant scopeScheduler`Scheduling\SyncScheduler`Cadence walker, `chunkById(100)`, active-only filterJob`ConnectorSyncJob``$tries=3`, exponential backoff, tenant-restore safetyModels`ConnectorInstallation` + `ConnectorCredential``BelongsToTenant` trait, cascade deleteMigrations`connector_installations` + `connector_credentials`Auto-loaded by the service providerExceptions`ConnectorAuthException`, `ConnectorApiException`, `ConnectorPaginationLimitException`, `RegistryConfigurationException`Distinct failure semantics: auth = no retry, api = retry, paginator-limit = partial successDTOs`SyncResult`, `HealthStatus`Immutable outcomesTenancy`Support\TenantContext` + `Models\Concerns\BelongsToTenant`Request-scoped tenant, auto-fill on creatingCredential form *(optional)*`Contracts\SupportsCredentialForm` + `Support\CredentialField`Opt-in interface for credential-based connectors (IMAP, API key, …) — host renders a native admin form instead of OAuth redirectFolder discovery *(optional, v1.4)*`Contracts\SupportsFolderDiscovery`Opt-in interface — `listAvailableFolders()` enumerates the live containers (IMAP folders, labels, spaces) an operator can whitelist; the connector owns auth + client lifecycleConnection settings *(optional, v1.4)*`Contracts\SupportsConnectionSettings` + `Support\CredentialField`Opt-in interface — `connectionSettingsSchema()` declares the editable post-install sync knobs (window, folders, filters), rendered by the host as a generic settings editorArchitecture at a glance ------------------------ [](#architecture-at-a-glance) ``` ┌──────────────────────────┐ │ Your connector package │ │ composer extra.askmydocs│ └─────────────┬────────────┘ │ auto-discovered ▼ ┌────────────────────┐ ┌──────────────────┐ ┌──────────────────────────┐ │ Cadence scheduler │ ──────▶ │ ConnectorRegistry│ ◀────── │ Host: config/connectors │ │ (every minute) │ │ R23 boot-time │ │ ::built_in (optional) │ └────────┬───────────┘ │ FQCN validation │ └──────────────────────────┘ │ └─────────┬────────┘ │ dispatch │ resolve by key() ▼ ▼ ┌────────────────────┐ ┌──────────────────┐ ┌──────────────────────────┐ │ ConnectorSyncJob │ ──────▶ │ Your connector │ ──────▶ │ OAuthCredentialVault │ │ tenant-restore │ │ syncIncremental()│ │ AES + lockForUpdate (R21)│ │ tries=3, backoff │ └─────────┬────────┘ └──────────────────────────┘ └────────────────────┘ │ │ fetches changed docs ▼ ┌────────────────────────┐ │ Host ingest pipeline │ │ (e.g. AskMyDocs │ │ IngestDocumentJob) │ └────────────────────────┘ ``` Installation ------------ [](#installation) ``` composer require padosoft/askmydocs-connector-base ``` The service provider is auto-discovered (Laravel package discovery). The package ships its own migrations — run them: ``` php artisan migrate ``` Want to copy the migrations into your host app's `database/migrations/` (e.g. to tweak `tenant_id` length)? Publish them: ``` php artisan vendor:publish --tag=connector-migrations ``` Same for the config: ``` php artisan vendor:publish --tag=connector-config ``` Wire the scheduler from your host app's `bootstrap/app.php`: ``` use Padosoft\AskMyDocsConnectorBase\Scheduling\SyncScheduler; ->withSchedule(function (Schedule $schedule): void { (new SyncScheduler)->registerSchedules($schedule); }) ``` That's it. Connector packages installed via composer are now auto-discovered and synced on cadence. Quick start — write your first connector in 50 lines ---------------------------------------------------- [](#quick-start--write-your-first-connector-in-50-lines) Create a new Laravel package. Add `padosoft/askmydocs-connector-base` to its `require`. Declare your connector class FQCN under `extra.askmydocs.connectors`: ``` // composer.json (your package) { "name": "you/askmydocs-connector-myapi", "require": { "padosoft/askmydocs-connector-base": "^1.0" }, "autoload": { "psr-4": { "You\\AskMyDocsConnectorMyApi\\": "src/" } }, "extra": { "askmydocs": { "connectors": [ "You\\AskMyDocsConnectorMyApi\\MyApiConnector" ] } } } ``` Implement the connector: ``` namespace You\AskMyDocsConnectorMyApi; use Carbon\Carbon; use Illuminate\Http\Request; use Padosoft\AskMyDocsConnectorBase\BaseConnector; use Padosoft\AskMyDocsConnectorBase\HealthStatus; use Padosoft\AskMyDocsConnectorBase\SyncResult; use Padosoft\AskMyDocsConnectorBase\Exceptions\ConnectorAuthException; final class MyApiConnector extends BaseConnector { public function key(): string { return 'my-api'; } public function displayName(): string { return 'My API'; } public function oauthScopes(): array { return ['read:docs']; } public function initiateOAuth(int $installationId): string { $state = $this->issueOAuthState($installationId); return 'https://my-api.example.com/oauth/authorize?state='.$state.'&...'; } public function handleOAuthCallback(int $installationId, Request $request): void { if (! $this->consumeOAuthState($installationId, (string) $request->query('state'))) { throw new ConnectorAuthException('Bad state'); } // Exchange code -> token, then: $this->vault->setCredentials($installationId, 'access-token', refreshToken: 'refresh'); } public function syncFull(int $installationId): SyncResult { return $this->syncIncremental($installationId, null); } public function syncIncremental(int $installationId, ?Carbon $since): SyncResult { // Fetch changed docs, dispatch host ingest jobs, count them. return new SyncResult( documentsAdded: 5, documentsUpdated: 2, documentsRemoved: 0, errors: [], completedAt: Carbon::now(), ); } public function disconnect(int $installationId): void { $this->vault->clearCredentials($installationId); } public function health(int $installationId): HealthStatus { return HealthStatus::healthy(); } } ``` `composer require you/askmydocs-connector-myapi` in any AskMyDocs install — the registry auto-discovers it, the scheduler starts dispatching it on cadence, the admin UI lists it in the available-connectors picker. The 10-method contract ---------------------- [](#the-10-method-contract) Every connector implements 10 methods (3 metadata + 1 scope + 2 OAuth + 2 sync + 1 disconnect + 1 health): MethodPurposeThrows`key(): string`Stable kebab-case identifier (`google-drive`, `notion`). Used as URL slug + `connector_installations.connector_name`.—`displayName(): string`Human label shown in the admin UI.—`iconUrl(): string`Connector logo URL. `BaseConnector` provides a default that resolves `public/connectors/{key}.svg` via `asset()`.—`oauthScopes(): array`List of scope strings the provider requires. Surfaced to the user in the install confirmation dialog.—`initiateOAuth(int): string`Build the provider's authorization URL. Use `$this->issueOAuthState()` for CSRF.`ConnectorAuthException``handleOAuthCallback(int, Request): void`Exchange auth code -> tokens, verify state, persist via `$this->vault->setCredentials()`.`ConnectorAuthException` on any failure`syncFull(int): SyncResult`Full discovery + ingestion. Long-running. Called at install + operator re-sync.propagates upstream errors`syncIncremental(int, ?Carbon): SyncResult`Delta since `$since`. Falls back to `syncFull` when `$since === null`. Called by the cadence scheduler.`ConnectorApiException` for transient (retry), `ConnectorAuthException` for credentials (no retry)`disconnect(int): void`Clear credentials, optionally revoke at provider.swallow / log; framework deletes installation row after`health(int): HealthStatus`Fast (under 2s) side-effect-free probe.returns `HealthStatus::errored(...)` instead of throwingOptional: credential form interface ----------------------------------- [](#optional-credential-form-interface) For connectors that use **credentials instead of OAuth** (IMAP, SMTP, API-key-based providers, ...), implement the optional `SupportsCredentialForm` interface alongside `ConnectorInterface`. The host detects the interface via `instanceof` at install time and renders a native admin form. Each field is described by a `CredentialField` value object — call `toArray()` on each to produce the JSON shape the host expects. ``` use Padosoft\AskMyDocsConnectorBase\Contracts\SupportsCredentialForm; use Padosoft\AskMyDocsConnectorBase\Support\CredentialField; final class ImapConnector extends BaseConnector implements SupportsCredentialForm { public function credentialFormSchema(): array { return [ (new CredentialField( name: 'host', label: 'IMAP Host', type: 'text', target: 'connection', required: true, ))->toArray(), (new CredentialField( name: 'port', label: 'Port', type: 'number', target: 'connection', required: true, default: 993, ))->toArray(), (new CredentialField( name: 'username', label: 'Username', type: 'text', target: 'connection', required: true, ))->toArray(), (new CredentialField( name: 'password', label: 'Password', type: 'password', target: 'secret', required: true, secret: true, // routed to vault, never stored in config_json ))->toArray(), ]; } } ``` **`CredentialField` properties:** PropertyTypeDescription`name``string`Form-data key (e.g. `'host'`)`label``string`Human-readable UI label`type``string``text` | `number` | `password` | `select` | `checkbox` | `multiselect` *(v1.4)* | `tags` *(v1.4)*`target``string``connection` | `config` → `config_json`; `auth_mode`; `provider`; `secret` → vault`required``bool`Whether the field must be filled`secret``bool`Masked in UI; routed to vault, never `config_json``default``mixed`Pre-filled value (a list for `multiselect`/`tags`)`options``array`For `select`/`multiselect` with a fixed set: `['value' => 'Label']``showIf``array{field:string,equals:string}|null`Conditional display rule`help``string|null`Helper text rendered below the field`group``string|null`Optional UI section heading`discovery` *(v1.4)*`string|null`For a live `multiselect`: names the discovery source (`'folders'` → `SupportsFolderDiscovery`)Connectors that use only the standard **OAuth redirect** do **not** implement this interface — it is entirely opt-in and backward compatible. Optional: folder discovery + editable settings (v1.4) ----------------------------------------------------- [](#optional-folder-discovery--editable-settings-v14) A connector that has selectable containers (IMAP folders, Gmail labels, …) and tunable sync behaviour opts into two more capabilities. The host detects each via `instanceof` (R23 — no connector-name branch) and renders a **generic, schema-driven settings editor** seeded with the installation's current `config_json` — no bespoke per-connector form. ``` use Padosoft\AskMyDocsConnectorBase\Contracts\SupportsConnectionSettings; use Padosoft\AskMyDocsConnectorBase\Contracts\SupportsFolderDiscovery; use Padosoft\AskMyDocsConnectorBase\Support\CredentialField; final class ImapConnector extends BaseConnector implements SupportsCredentialForm, SupportsFolderDiscovery, SupportsConnectionSettings { /** Live container list — the connector owns auth + client lifecycle. */ public function listAvailableFolders(int $installationId): array { $client = $this->makeClient($installationId); // handles basic + xoauth2 token refresh try { return $client->listMailboxes(); // list, verbatim } finally { $client->close(); } } /** Editable post-install sync knobs — every field target='config', never secret. */ public function connectionSettingsSchema(): array { return [ (new CredentialField( name: 'folders.include', label: 'Folders to sync', type: 'multiselect', target: 'config', default: [], group: 'Folders', discovery: 'folders', help: 'Empty = sync every non-excluded folder.', ))->toArray(), (new CredentialField( name: 'folders.exclude', label: 'Folders to skip', type: 'multiselect', target: 'config', default: [], group: 'Folders', discovery: 'folders', ))->toArray(), (new CredentialField( name: 'date_window_days', label: 'Sync window (days)', type: 'number', target: 'config', default: 365, group: 'Sync window', help: '0 = all history.', ))->toArray(), (new CredentialField( name: 'senders.exclude', label: 'Exclude senders', type: 'tags', target: 'config', default: [], group: 'Filtering', ))->toArray(), ]; } } ``` The settings field `name` is a **dotted path** the host writes into `config_json` (`folders.include` → `config_json['folders']['include']`) — exactly what the connector reads back at sync time, so a picked value round-trips 1:1. Both capabilities are opt-in and backward compatible. How auto-discovery works ------------------------ [](#how-auto-discovery-works) `ConnectorRegistry` merges two sources at boot: 1. **`config/connectors.php::built_in`** — FQCN list for connectors the host app wires by hand (rare). 2. **`composer.lock` packages** — every entry whose `extra.askmydocs.connectors` is a non-empty array of FQCNs. Each FQCN is resolved through the container and `instanceof`-checked against `ConnectorInterface` (R23). Failure modes: - Class missing -> `RegistryConfigurationException: '...' does not exist` - Class exists but doesn't implement -> `RegistryConfigurationException: '...' does not implement ConnectorInterface` - Two connectors return the same `key()` -> `RegistryConfigurationException: Duplicate connector key '...'` - Container can't instantiate -> `RegistryConfigurationException: '...' could not be resolved` All boot-time. No silent fallthrough to a confusing "undefined method" later. Credential vault — encrypted, atomic, tenant-scoped --------------------------------------------------- [](#credential-vault--encrypted-atomic-tenant-scoped) `OAuthCredentialVault` is the single chokepoint for every connector's tokens: - **AES-256 encryption at rest** via Laravel `Crypt`. The DB row never sees plaintext. - **Tenant-scoped reads** — every query joins to `connector_installations` and filters by the active `TenantContext`. Cross-tenant reads return `null`, not the wrong tenant's tokens. - **Refresh-aware** — `getAccessToken()` returns `null` for expired tokens. Connectors call `getRefreshToken()` to rotate via the provider's `/oauth2/token` endpoint, then `setCredentials()` to persist the rotated pair. - **R21 — atomic `setExtraKey`** — concurrent writers updating different keys in `extra_json` (e.g. one connector storing `bot_id`, another storing `changes_page_token`) MUST NOT race. Implementation: ``` DB::transaction(function () use (...) { $row = ConnectorCredential::query() ->where(...) ->lockForUpdate() // SELECT ... FOR UPDATE ->first(); if ($row === null) { throw new ConnectorAuthException('credential row was deleted concurrently'); } $extra = $row->extra_json ?? []; $extra[$key] = $value; $row->extra_json = $extra; $row->save(); // same transaction }); ``` A read-modify-write without the lock loses siblings under contention. The package was extracted from AskMyDocs precisely after this race was caught + fixed in production. Scheduler + sync job -------------------- [](#scheduler--sync-job) `SyncScheduler::registerSchedules($schedule)` registers one `everyMinute()` closure. The closure walks every `STATUS_ACTIVE` installation in `chunkById(100)` and dispatches `ConnectorSyncJob` for each that's due (i.e. `last_sync_at + cadenceMinutes