agek-tileserverdata (0.1.0)

Converts SLPK/TPKX/IFC/RVT/LAS/raster/mesh packages uploaded to an S3 input bucket into i3sREST or tile format in an output bucket, and (optionally) publishes the result as an ArcGIS Enterprise service item. Built for both the browser SPA and machine-to-machine callers (FME and similar) — both use the exact same endpoints and header scheme below.

Every endpoint except the health probes requires:

X-Client-Id/X-Client-Secret are issued by Geodata per customer (a dedicated entry in the customer's client config — a machine-to-machine caller like FME gets its own, separate from the browser SPA's) — ask Geodata to provision one, it isn't self-service in Portal. X-Portal-Token is self-service — see the next section.

A browser SPA gets this from its OAuth2 login. A script/scheduled job (FME or similar) has no browser, so it needs one of these instead — all three produce a value that's used exactly the same way, unchanged, as X-Portal-Token:

Setting up an ArcGIS API key in Portal

This is the recommended option for unattended production automation (no password to rotate, no login session to keep alive) — requires ArcGIS Enterprise 10.9.1+.

1. Sign in to the Portal as the named user the integration should act as (needs publishing rights to whatever the integration will publish).
2. Go to Content → New item → Developer credential (menu wording varies a little by Portal version — look for "API key" / "Developer credential" under Content or My Content if it isn't exactly there).
3. Choose the privileges the key needs (at minimum: publish content) and, optionally, an expiration and HTTP referrer restriction.
4. Create it and copy the key value it gives you — Portal shows it once.
5. Send that value as X-Portal-Token on every API call. Store it the same way you'd store a password/secret — anyone with it can act as that user.

generateToken example (FME HTTP Caller, curl, or similar)

POST {portal_url}/sharing/rest/generateToken
Body: username=svc_fme&password=...&client=requestip&expiration=120&f=json
→ {"token": "<portal-token>", "expires": ...}

OAuth Credentials example (client_credentials grant)

POST {portal_url}/sharing/rest/oauth2/token
Body: client_id=<oauth_credentials_client_id>&client_secret=<oauth_credentials_client_secret>&grant_type=client_credentials&f=json
→ {"access_token": "<portal-token>", "expires_in": ...}

Long-running jobs and token expiry: extraction can take from minutes to a few hours for large files, but X-Portal-Token is captured once, at the moment you call /publish or /uploads/complete, and frozen for the whole job — there is no server-side refresh for a machine caller. If the token expires before extraction finishes, the publish step fails with an invalid-token error even though the file processed fine. Request an expiration (or API-key expiry) comfortably longer than your largest expected file's processing time, not just long enough to make the initial call. A browser SPA can instead pass an optional portal_refresh_token alongside auto_publish so the backend mints a fresh token right before publishing — that field is SPA-specific (a real OAuth refresh token isn't something generateToken/API-key/client_credentials produce) and, being a credential, must never be logged or displayed.

Two independent upload mechanisms exist — pick per file size / integration:

• Proxy (POST /upload) — the file streams through this API. Simple (one endpoint), but the whole file passes through the API pod and its self-imposed ingress timeout, so it doesn't suit very large files. See "Proxy upload" below.
• Presigned (POST /uploads + friends) — the client PUTs bytes straight to S3; this API only ever signs URLs and never sees file bytes. Built for large files. See "Presigned direct-to-S3 upload" below.

Both eventually enqueue the exact same background extraction+publish job — only the path the bytes take to reach S3 differs.

Two ways to run the full pipeline over the proxy path — pick whichever shape suits the integration. Both end up enqueuing the exact same background job with the same parameters.

Option A — one call

POST /upload (multipart form) with auto_publish=true, service_name, and X-Portal-Token set does everything: streams the file to S3, then immediately enqueues extraction + publish. The response is a Server-Sent-Events stream:

data: {"status": "uploading", "bytes_uploaded": 1048576, "total": 52428800, "progress_pct": 2.0}
...
data: {"status": "uploaded", "input_key": "tileserver/input/model.slpk"}
data: {"status": "queued", "task_id": "abc123", "task_name": "process_slpk", "poll_url": "/tasks/abc123"}

Take task_id from the final queued event and poll GET /tasks/{task_id}.

Option B — two calls (upload, then publish separately)

Useful for confirming the upload succeeded before committing to a publish, letting a user pick the service name/sharing afterwards, or re-publishing an already-uploaded file.

1. POST /upload with auto_publish omitted (defaults to false) — same SSE stream as above, but it stops after the uploaded event. Keep the input_key from that event.
2. POST /publish with {"portal_url", "input_key", "service_name", ...} and X-Portal-Token — enqueues the same extraction+publish job. Responds immediately with a TaskModel (id / poll_url), as plain JSON rather than a stream.

Four calls, three of them to this API and one straight to S3 — no file bytes ever pass through this API:

1. POST /uploads — {"portal_url", "filename", "size"} → {"upload_id", "input_key", "part_size"}. Starts an S3 multipart upload and tells you the part size to slice the file into (currently 64 MiB).
2. POST /uploads/parts — {"portal_url", "input_key", "upload_id", "part_numbers": [1, 2, ...]} (up to 100 part numbers per call) → a presigned S3 URL per part number, each valid for 1 hour.
3. PUT each file part directly to its presigned URL — not a call to this API at all. S3 returns an ETag header on each successful PUT; keep every (part_number, etag) pair. Re-sign a part (repeat step 2 for just that number) if its URL expires or the PUT fails and needs retrying past the 1-hour window. A browser client's CORS preflight for these PUTs is handled by the S3 bucket's CORS config, not this API.
4. POST /uploads/complete — {"portal_url", "input_key", "upload_id", "parts": [{"part_number", "etag"}, ...], "auto_publish", "service_name", ...} and (if publishing) X-Portal-Token — finalizes the S3 multipart upload and, if auto_publish=true, enqueues the same extraction+publish job as the proxy path. Responds immediately with {"input_key", "task"} (JSON, task is null unless auto_publish=true).

Abandoning an upload partway through: DELETE /uploads/{upload_id}?portal_url=...&input_key=... aborts the S3 multipart upload so it stops counting toward storage. Skipping this isn't catastrophic — incomplete multipart uploads are cleaned up automatically after a few days — but it's the clean way to cancel.

GET /tasks/{task_id} works for a task from either option above. Poll until status is SUCCESS or FAILURE; result then holds either the published service info (PackageResult: service_url, item_id, ...) or a FailureResult.

.slpk .tpkx .las/.laz/.zlas .tif/.tiff/.img/.ecw .ifc/.rvt .osgb/.obj/.dae. An .ifc/.rvt may also be uploaded as a .zip bundle together with georeferencing sidecars (.prj/.wld3) — the model file inside determines the task.

POST /tasks/process_slpk / POST /tasks/process_tpkx and their matching GET endpoints are deprecated — use POST /publish and GET /tasks/{task_id} instead, which work for every supported file type.

Proxy upload: send a file through this API to S3 and run it through extraction + publish, in one call or two. See the description above for the full sequence.

Upload straight to S3 via presigned URLs — this API only signs URLs, file bytes never pass through it. Recommended for large files. See the description above for the 4-step sequence.

Look up past jobs and fetch a permanent link back to an originally-uploaded source file.

Liveness/readiness probes and a startup-config diagnostic view — not part of the integration flow.