# Meyalabs Health — Production Readiness Runbook

This runbook is the operational gate after the Phase 1 P0 code suite passes. A deployment is not production-ready until every required item below is verified in the target environment.

## 1. Runtime contract

- PHP 8.4 or newer with the extensions required by Laravel, MySQL, encryption, HTTP clients, and image/file validation.
- MySQL 8 with strict mode, InnoDB, UTC database timestamps, automated backups, and point-in-time recovery where the provider supports it.
- HTTPS only. Redirect HTTP to HTTPS at the reverse proxy and send HSTS after the production domain is confirmed.
- A persistent queue worker. `sync` is forbidden in production.
- Laravel scheduler invoked every minute.
- Writable `storage/framework`, `storage/logs`, and configured public/media storage.
- Server clock synchronized with NTP. Application timezone remains `Asia/Jakarta`.

Start from `.env.production.example`. Secret values must be injected by the deployment platform and must not be committed.

## 2. Pre-deployment gate

Run against a restored copy of the latest production backup first:

```text
composer install --no-dev --prefer-dist --optimize-autoloader
npm ci
npm run build
php artisan migrate --pretend
php artisan test --configuration=phpunit.mysql.xml
```

Review every migration shown by `migrate --pretend`. The Phase 1 changes are additive; do not run manual `DROP`, `TRUNCATE`, or destructive data rewrites.

Create and verify a recoverable database backup immediately before deployment. A backup is only considered valid after a restore drill into an isolated database succeeds.

## 3. Deployment sequence

```text
php artisan down --render="errors::503" --retry=60
php artisan migrate --force
php artisan storage:link
php artisan optimize
php artisan queue:restart
php artisan up
```

For zero-downtime release systems, switch the release symlink only after dependencies, assets, migration, and cache generation complete successfully.

Do not roll back database migrations automatically. Roll back the application release only when its previous version remains compatible with the additive schema; otherwise ship a forward fix.

## 4. Required long-running processes

Queue worker baseline:

```text
php artisan queue:work database --queue=default --sleep=1 --tries=3 --timeout=90 --max-time=3600
```

Use systemd, Supervisor, a container orchestrator, or the hosting provider's process manager to restart failed workers automatically.

Scheduler entry, once per minute:

```text
php artisan schedule:run
```

The scheduler writes `ops:scheduler:last_heartbeat`. `/health/ready` returns HTTP 503 in production if that heartbeat is older than three minutes.

## 5. Health and monitoring

- `/up`: process liveness.
- `/health/ready`: database, application key, debug mode, async queue, queue tables, scheduler heartbeat, writable storage, payment gateway, and shipping gateway readiness.
- Alert on HTTP 5xx rate, queue failures, stale jobs, scheduler heartbeat failure, payment webhook failures, shipping reconciliation, stock reservation failures, refund failures, and settlement discrepancies.
- Use daily structured application logs with at least 30 days retention. Forward warning-and-higher events to centralized monitoring.
- Review `failed_jobs` continuously and alert when it is non-empty for more than five minutes.

The readiness response intentionally exposes booleans only and never credentials.

## 6. Payment production gate

- Use `PAYMENT_PROVIDER=midtrans`; `demo` makes readiness fail in production.
- Configure production server key, client key, merchant ID, and production API URLs.
- Register the exact HTTPS webhook URL and verify signatures end-to-end.
- Execute real low-value transactions for settlement, duplicate settlement, expiry, cancellation, invalid signature, amount mismatch, and delayed/out-of-order callbacks.
- Reconcile the gateway transaction against one `PlatformOrder`, one `Payment`, all sibling `SellerOrder` records, stock movements/reservations, ledger entries, and shipment jobs.

## 7. Shipping production gate

- Use `SHIPPING_PROVIDER=biteship`; `demo` makes readiness fail in production.
- Configure a high-entropy webhook token and validate the configured header at the provider.
- Validate quote, booking, duplicate booking, ordinary failure retry, ambiguous timeout, webhook replay, out-of-order delivery event, and reconciliation workflow.
- A shipment in `reconciliation_required` must be resolved manually after checking the provider dashboard. Never blindly retry it.

## 8. Data protection and recovery

- Automated encrypted database backup at least daily; transaction-log/PITR retention is recommended for payment workloads.
- Encrypted off-host copy with documented retention and deletion policy.
- Monthly restore drill into an isolated environment.
- Object/media storage versioning or equivalent recovery.
- Record recovery time objective and recovery point objective before launch.
- Rotate application, database, payment, shipping, OAuth, mail, and webhook secrets using the platform's secret manager.

Changing `APP_KEY` without a controlled key-rotation plan will make encrypted fields unreadable. Back it up separately and restrict access.

## 9. Closed-pilot acceptance

Before onboarding external pharmacies, run a closed pilot and verify:

- one-, two-, and three-pharmacy checkout;
- platform and merchant vouchers;
- different shipping costs;
- prescription and non-prescription combinations;
- payment paid, pending, failed, expired, duplicate, and out-of-order callbacks;
- full seller refund and item partial refund;
- shipment failure and retry;
- expired merchant and pharmacist credentials;
- cross-tenant IDOR attempts;
- stock-last-unit concurrency;
- settlement totals and withdrawal eligibility.

Track checkout success, payment success, stock mismatch, shipment failure, refund rate, cancellation, webhook errors, settlement discrepancy, authorization denials, and prescription SLA.

## 10. Go-live decision

Go live only when:

1. the full MySQL regression suite is green;
2. `/health/ready` returns 200 in the production environment after the scheduler and queue worker are active;
3. backup restoration has been demonstrated;
4. Midtrans and Biteship production credentials and callbacks have been validated;
5. monitoring and on-call ownership are active;
6. the business owner accepts the compliance matrix and closed-pilot results.

Items 3–6 require the production infrastructure, provider accounts, and business operations. They cannot be proven by local source-code tests alone.