Configure OCI Regional WAF Rate Limiting with a Custom 503 Response Using OCI CLI

Configure OCI Regional WAF Rate Limiting with a Custom 503 Response Using OCI CLI

Overview

This guide shows how to configure an Oracle Cloud Infrastructure regional Web Application Firewall policy to:

  • Monitor requests to eits-uat.fcicanada.com.
  • Apply rate limiting only to requests whose path begins with /apex/.
  • Limit each client IP address to 100 requests within 60 seconds.
  • Return a custom HTTP 503 page for 300 seconds when the threshold is exceeded.
  • Preserve all existing WAF actions and rules.

The tested regional WAF policy was:

Policy name: QA_RegionalWAF_Policy
Region: ca-toronto-1
Lifecycle state: ACTIVE
Target host: eits-uat.fcicanada.com
Target path: /apex/
Important: OCI WAF rate limiting is evaluated per client IP address. It does not calculate one combined request total for all application users. Users behind the same NAT gateway, VPN, proxy, or secure web gateway may share one visible public IP address.

Prerequisites

  • OCI CLI installed and configured.
  • An OCI CLI profile with permission to read and update the WAF policy.
  • jq installed.
  • The regional WAF policy OCID.
  • A Linux shell such as Bash.

Step 1: Create a Working Directory

mkdir -p rate_limiting_waf
cd rate_limiting_waf

Step 2: Set Environment Variables

Set the OCI profile, region, WAF policy OCID, action name, and rule name.

export OCI_PROFILE="CLOUD_ADMIN_FCIAS2"
export OCI_REGION="ca-toronto-1"

export WAF_POLICY_ID="ocid1.webappfirewallpolicy.oc1.ca-toronto-1.REPLACE_WITH_POLICY_OCID"

export ACTION_NAME="EITS-UAT-HIGH-VOLUME-503"
export RULE_NAME="EITS-UAT-APEX-RATE-LIMIT"
Security note: Do not publish a production or internal OCID in a public blog post. Replace it with a placeholder as shown above.

Step 3: Verify the WAF Policy

oci waf web-app-firewall-policy get \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --query 'data.{"Name":"display-name","State":"lifecycle-state","OCID":id}' \
  --output table

Example output:

+-----------------------+----------------------------------------------+--------+
| Name                  | OCID                                         | State  |
+-----------------------+----------------------------------------------+--------+
| QA_RegionalWAF_Policy | ocid1.webappfirewallpolicy...                | ACTIVE |
+-----------------------+----------------------------------------------+--------+

Confirm that the policy is the correct one and that its lifecycle state is ACTIVE.

Step 4: Back Up the Existing WAF Policy

The OCI CLI update command submits complete policy sections. Back up the policy before changing the actions or request-rate-limiting arrays.

oci waf web-app-firewall-policy get \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  > waf-policy-before-rate-limit.json

Extract the existing actions and rate-limiting configuration:

jq '.data.actions' \
  waf-policy-before-rate-limit.json \
  > actions-before.json

jq '.data."request-rate-limiting"' \
  waf-policy-before-rate-limit.json \
  > request-rate-limiting-before.json

In this implementation, the original rate-limiting section returned:

null

That confirmed that no request rate-limiting rules were configured before this change.

Step 5: Create the Custom 503 HTML Page

Create the static response page that OCI WAF will return when a client exceeds the configured request threshold.

cat > eits-high-volume.html <<'EOF'
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>EITS Intake – Temporarily Unavailable</title>
    <style>
        body {
            margin: 0;
            min-height: 100vh;
            display: flex;
            align-items: center;
            justify-content: center;
            background: #f5f6f7;
            color: #222;
            font-family: Arial, Helvetica, sans-serif;
        }

        .message {
            max-width: 700px;
            margin: 20px;
            padding: 40px;
            text-align: center;
            background: #fff;
            border-radius: 8px;
            box-shadow: 0 3px 15px rgba(0, 0, 0, 0.12);
        }

        h1 {
            margin-top: 0;
            font-size: 28px;
        }

        p {
            font-size: 18px;
            line-height: 1.6;
        }
    </style>
</head>
<body>
    <main class="message">
        <h1>503 Service Temporarily Unavailable</h1>

        <p>
            EITS Intake is currently experiencing a high volume of requests
            and is unable to process your request at this time.
        </p>

        <p>
            Please wait a few minutes and try again. We apologize for the
            inconvenience and appreciate your patience.
        </p>
    </main>
</body>
</html>
EOF
Correction: The tested shell output contained <h1>503 Service Temporarily Unavailable /h1>. The version above corrects it to a valid closing tag: </h1>.

Step 6: Build the Complete WAF Actions JSON

The following command reads the current action list, removes any older action with the same name, and appends the custom RETURN_HTTP_RESPONSE action.

jq \
  --rawfile html eits-high-volume.html \
  --arg actionName "$ACTION_NAME" \
  '
  (.data.actions // [])
  | map(select(.name != $actionName))
  + [
      {
        "name": $actionName,
        "type": "RETURN_HTTP_RESPONSE",
        "code": 503,
        "headers": [
          {
            "name": "Content-Type",
            "value": "text/html; charset=UTF-8"
          },
          {
            "name": "Retry-After",
            "value": "300"
          },
          {
            "name": "Cache-Control",
            "value": "no-store"
          }
        ],
        "body": {
          "type": "STATIC_TEXT",
          "text": $html
        }
      }
    ]
  ' waf-policy-before-rate-limit.json > actions.json

Review the generated JSON:

jq . actions.json

The new action should include:

{
  "name": "EITS-UAT-HIGH-VOLUME-503",
  "type": "RETURN_HTTP_RESPONSE",
  "code": 503,
  "headers": [
    {
      "name": "Content-Type",
      "value": "text/html; charset=UTF-8"
    },
    {
      "name": "Retry-After",
      "value": "300"
    },
    {
      "name": "Cache-Control",
      "value": "no-store"
    }
  ],
  "body": {
    "type": "STATIC_TEXT",
    "text": "..."
  }
}

Step 7: Add the Custom Action to the WAF Policy

oci waf web-app-firewall-policy update \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --actions file://actions.json \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --wait-for-state SUCCEEDED \
  --force

Successful output includes:

"operation-type": "UPDATE_WAF_POLICY"
"percent-complete": 100.0
"status": "SUCCEEDED"

Step 8: Verify the Custom Action

oci waf web-app-firewall-policy get \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --query "data.actions[?name=='${ACTION_NAME}']" \
  --output json

Confirm that the returned action has:

  • type: RETURN_HTTP_RESPONSE
  • code: 503
  • Content-Type: text/html; charset=UTF-8
  • Retry-After: 300
  • Cache-Control: no-store

Step 9: Retrieve the Policy Again

Retrieve the policy after the action has been added. This refreshed JSON will be used to preserve the current configuration while adding the rate-limiting rule.

oci waf web-app-firewall-policy get \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  > waf-policy-with-action.json

Step 10: Build the Rate-Limiting Rule

The rule below applies only when both of these conditions are true:

  • The HTTP host is eits-uat.fcicanada.com.
  • The request path begins with /apex/.

It does not match a specific APEX session identifier because values such as 10395221140358 change between sessions.

jq \
  --arg ruleName "$RULE_NAME" \
  --arg actionName "$ACTION_NAME" \
  '
  {
    "rules":
      (
        (.data."request-rate-limiting".rules // [])
        | map(select(.name != $ruleName))
        + [
            {
              "name": $ruleName,
              "type": "REQUEST_RATE_LIMITING",
              "conditionLanguage": "JMESPATH",
              "condition": "http.request.host == '\''eits-uat.fcicanada.com'\'' && starts_with(http.request.url.path, '\''/apex/'\'')",
              "actionName": $actionName,
              "configurations": [
                {
                  "requestsLimit": 100,
                  "periodInSeconds": 60,
                  "actionDurationInSeconds": 300
                }
              ]
            }
          ]
      )
  }
  ' waf-policy-with-action.json > request-rate-limiting.json

Review the generated rate-limiting configuration:

jq . request-rate-limiting.json

Expected output:

{
  "rules": [
    {
      "name": "EITS-UAT-APEX-RATE-LIMIT",
      "type": "REQUEST_RATE_LIMITING",
      "conditionLanguage": "JMESPATH",
      "condition": "http.request.host == 'eits-uat.fcicanada.com' && starts_with(http.request.url.path, '/apex/')",
      "actionName": "EITS-UAT-HIGH-VOLUME-503",
      "configurations": [
        {
          "requestsLimit": 100,
          "periodInSeconds": 60,
          "actionDurationInSeconds": 300
        }
      ]
    }
  ]
}

Step 11: Apply the Rate-Limiting Rule

oci waf web-app-firewall-policy update \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --request-rate-limiting file://request-rate-limiting.json \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --wait-for-state SUCCEEDED \
  --force

Successful output includes:

"operation-type": "UPDATE_WAF_POLICY"
"percent-complete": 100.0
"status": "SUCCEEDED"

Step 12: Verify the Rate-Limiting Rule

oci waf web-app-firewall-policy get \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --query "data.\"request-rate-limiting\".rules[?name=='${RULE_NAME}']" \
  --output json

Verified output:

[
  {
    "action-name": "EITS-UAT-HIGH-VOLUME-503",
    "condition": "http.request.host == 'eits-uat.fcicanada.com' && starts_with(http.request.url.path, '/apex/')",
    "condition-language": "JMESPATH",
    "configurations": [
      {
        "action-duration-in-seconds": 300,
        "period-in-seconds": 60,
        "requests-limit": 100
      }
    ],
    "name": "EITS-UAT-APEX-RATE-LIMIT",
    "type": "REQUEST_RATE_LIMITING"
  }
]

Step 13: List All WAF Actions

oci waf web-app-firewall-policy get \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --query 'data.actions[].{Name:name,Type:type}' \
  --output table

Verified output:

+-----------------------------------------+----------------------+
| Name                                    | Type                 |
+-----------------------------------------+----------------------+
| Pre-configured Check Action             | CHECK                |
| Pre-configured Allow Action             | ALLOW                |
| Pre-configured 401 Response Code Action | RETURN_HTTP_RESPONSE |
| Unauthorized to access this web site    | RETURN_HTTP_RESPONSE |
| maintenance-503                         | RETURN_HTTP_RESPONSE |
| EITS-UAT-HIGH-VOLUME-503                | RETURN_HTTP_RESPONSE |
+-----------------------------------------+----------------------+

How the Configuration Works

Client request
      |
      v
OCI Regional WAF
      |
      |-- Host is not eits-uat.fcicanada.com
      |       '-- Rate-limit rule does not apply
      |
      |-- Path does not begin with /apex/
      |       '-- Rate-limit rule does not apply
      |
      '-- Host and path match
              |
              |-- Up to 100 requests within 60 seconds
              |       '-- Request continues to the reverse proxy
              |
              '-- Threshold exceeded for that client IP
                      '-- Return custom HTTP 503 page for 300 seconds

Configuration Values

Setting Value Meaning
requestsLimit 100 Maximum requests allowed from one client IP during the evaluation period.
periodInSeconds 60 The request-counting window.
actionDurationInSeconds 300 How long the custom response action remains active for the client IP after the threshold is exceeded.
Retry-After 300 Advises the client to retry after five minutes.

Testing

Use a non-production client and a safe URL. Avoid sending repeated requests to an APEX form submission endpoint.

seq 1 150 | xargs -n1 -P20 \
curl -sk -o /dev/null \
-w '%{http_code}\n' \
'https://eits-uat.fcicanada.com/apex/'

Expected behavior:

  • Initial requests receive the normal application response.
  • After the client IP exceeds the threshold, responses change to HTTP 503.
  • The custom HTML message is displayed.
  • The client IP remains subject to the action for 300 seconds.

To inspect response headers:

curl -skI https://eits-uat.fcicanada.com/apex/

During rate limiting, the response should include:

HTTP/1.1 503 Service Unavailable
Content-Type: text/html; charset=UTF-8
Retry-After: 300
Cache-Control: no-store

Rollback

Restore the Previous Actions

oci waf web-app-firewall-policy update \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --actions file://actions-before.json \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --wait-for-state SUCCEEDED \
  --force

Remove the Rate-Limiting Rule

Because the original rate-limiting configuration was null, use an empty rule list to remove the rule:

cat > request-rate-limiting-empty.json <<'EOF'
{
  "rules": []
}
EOF

oci waf web-app-firewall-policy update \
  --web-app-firewall-policy-id "$WAF_POLICY_ID" \
  --request-rate-limiting file://request-rate-limiting-empty.json \
  --profile "$OCI_PROFILE" \
  --region "$OCI_REGION" \
  --wait-for-state SUCCEEDED \
  --force
Rollback warning: Always retrieve and review the current policy immediately before rollback. Another administrator may have added valid actions or rules after the original backup was created.

Operational Considerations

  • Shared public IP addresses: Users behind Skyhigh, corporate NAT, VPN, or proxy infrastructure may appear as one client IP. Choose the request threshold using observed production traffic.
  • APEX sessions: Do not include the changing APEX session number in the WAF condition.
  • HTTP status: HTTP 503 is appropriate because the restriction represents temporary unavailability. HTTP 502 should normally be reserved for an invalid upstream gateway response.
  • Policy replacement behavior: The CLI update sends the entire actions or rate-limiting section. Always merge with the existing policy instead of submitting only the new object.
  • Monitoring: Review WAF logs after activation to confirm the rule is not affecting legitimate users.

Final Result

The regional WAF policy now protects the EITS UAT APEX endpoint with a per-client-IP request limit. A client that sends more than 100 matching requests within 60 seconds receives a custom HTTP 503 response for 300 seconds. Existing WAF actions remain preserved.

Setting Up Statspack on Oracle 19c Standard Edition: 15-Minute Snapshots with 7-Day Retention

 


If you're running Oracle 19c Standard Edition, you don't get access to AWR, ADDM, or ASH — those are part of the Diagnostics and Tuning Packs, which are licensed Enterprise Edition add-ons. Querying DBA_HIST_* views or calling dbms_workload_repository on an SE database isn't just unsupported, it's a license compliance risk that auditors will flag immediately.

The good news: Statspack has shipped with every Oracle release since 8i, it's fully included in SE at no extra cost, and it still works perfectly well on 19c. It captures point-in-time snapshots of V$ performance views — SQL execution stats, wait events, latches, segment stats — into a dedicated PERFSTAT schema, so you can diff two snapshots and get a report that does the same job as an AWR report.

This post walks through a production-ready setup for a common requirement: a snapshot every 15 minutes, with a 7-day rolling retention window. At that cadence you're looking at 96 snapshots a day, or roughly 672 snapshots in the repository once it reaches steady state — so sizing and automated purging matter from day one.

How Statspack Actually Works

Three moving parts:

The repository — a dedicated tablespace and schema (PERFSTAT) holding around 70 STATS$* tables, including STATS$SNAPSHOT, STATS$SQL_SUMMARY, STATS$SYSTEM_EVENT, and STATS$SEG_STAT.

Snapshot levels — these control how much data gets captured per snap:

  • Level 0: basic performance data only
  • Level 5 (the default, and the right choice for 15-minute intervals): adds SQL statements above resource thresholds
  • Level 7: adds segment-level statistics — useful when actively chasing hot blocks or I/O contention, but heavier
  • Level 10: adds child latch statistics — rarely needed, and real I/O overhead on busy systems

Scheduling and purging — older guides point you to dbms_job via spauto.sql, but on 19c you should use DBMS_SCHEDULER instead. It's more robust, has built-in retry logic, and it's already the standard mechanism for every other piece of database automation you're running.

Given a 15-minute interval, start at Level 5. Level 7 at that frequency on a busy OLTP system — especially something financial or healthcare-related — can generate noticeable shared-pool churn. Bump to Level 7 temporarily during an active investigation, then drop back down.

Step-by-Step Installation

1. Pre-checks

-- Confirm job queue processes are enabled (required for DBMS_SCHEDULER)
SHOW PARAMETER job_queue_processes;

-- Confirm Statspack isn't already installed
SELECT username FROM dba_users WHERE username = 'PERFSTAT';

2. Create a dedicated tablespace

Keep PERFSTAT out of SYSAUX or USERS. Isolating it makes space management and eventual removal far easier.

CREATE TABLESPACE perfstat
  DATAFILE '/u02/oradata/ORCL/perfstat01.dbf' SIZE 500M
  AUTOEXTEND ON NEXT 100M MAXSIZE 4G
  EXTENT MANAGEMENT LOCAL
  SEGMENT SPACE MANAGEMENT AUTO;

At 15-minute intervals with 7-day retention, expect the repository to land somewhere in the low hundreds of MB up to 1–2GB depending on SQL diversity on the instance. The 500M start / 4G ceiling above is a safe, conservative starting point — check actual growth after week one and adjust.

3. Run the install script as SYS

sqlplus / as sysdba
SQL> @?/rdbms/admin/spcreate.sql

It will interactively prompt for the PERFSTAT password (use something strong — this account is never meant to be logged into interactively), the default tablespace (perfstat), and a temporary tablespace (your existing TEMP).

Check the spool files — spcpkg.lis, spctab.lis, spcusr.lis — for any ORA- errors. A clean install ends with the package compiling without issue.

4. Grant CREATE JOB and verify

GRANT CREATE JOB TO perfstat;

-- Sanity check: should be roughly 70 tables
SELECT COUNT(*) FROM dba_tables WHERE owner = 'PERFSTAT';

5. Take a manual baseline snapshot

CONNECT perfstat/<password>
EXEC statspack.snap(i_snap_level => 5);
SELECT snap_id, snap_time FROM stats$snapshot ORDER BY snap_id;

Confirm this runs cleanly before automating anything.

6. Schedule snapshots every 15 minutes

BEGIN
  DBMS_SCHEDULER.CREATE_JOB (
    job_name        => 'STATSPACK_SNAPSHOT',
    job_type        => 'PLSQL_BLOCK',
    job_action      => 'BEGIN statspack.snap(i_snap_level => 5); END;',
    start_date      => SYSTIMESTAMP,
    repeat_interval => 'FREQ=MINUTELY;INTERVAL=15',
    enabled         => TRUE,
    comments        => 'Statspack snapshot every 15 minutes - level 5'
  );
END;
/

7. Schedule the purge job to enforce 7-day retention

BEGIN
  DBMS_SCHEDULER.CREATE_JOB (
    job_name        => 'STATSPACK_PURGE',
    job_type        => 'PLSQL_BLOCK',
    job_action      => 'BEGIN statspack.purge(i_purge_before_date => SYSDATE - 7, i_extended_purge => TRUE); END;',
    start_date      => SYSTIMESTAMP,
    repeat_interval => 'FREQ=DAILY;BYHOUR=2;BYMINUTE=30',
    enabled         => TRUE,
    comments        => 'Purge statspack snapshots older than 7 days'
  );
END;
/

i_extended_purge => TRUE matters here — it cleans up the SQL text and segment rows tied to purged snapshots, not just the STATS$SNAPSHOT headers. Skip it and you'll end up with orphaned rows in tables like STATS$SQL_SUMMARY, and the repository won't shrink the way you'd expect. Schedule the purge for a genuinely quiet window — 2:30 AM above is just an example; pick whatever fits your environment's actual lull after batch processing.

Validating the Setup

-- Confirm both jobs exist and are scheduled correctly
SELECT job_name, state, repeat_interval, next_run_date
FROM   dba_scheduler_jobs
WHERE  owner = 'PERFSTAT';

-- Confirm jobs are actually firing (check after 30-60 minutes)
SELECT job_name, status, actual_start_date, run_duration
FROM   dba_scheduler_job_run_details
WHERE  owner = 'PERFSTAT'
ORDER  BY actual_start_date DESC;

-- Confirm snapshot cadence is roughly 15 minutes apart
SELECT snap_id, snap_time
FROM   stats$snapshot
ORDER  BY snap_id DESC
FETCH FIRST 10 ROWS ONLY;

-- Confirm retention is holding at 7 days
SELECT MIN(snap_time) AS oldest_snap, MAX(snap_time) AS newest_snap,
       COUNT(*) AS total_snapshots
FROM   stats$snapshot;

-- Track tablespace growth weekly
SELECT tablespace_name,
       ROUND(SUM(bytes)/1024/1024,1) AS allocated_mb
FROM   dba_data_files
WHERE  tablespace_name = 'PERFSTAT'
GROUP BY tablespace_name;

Once the purge job has run a few cycles, total_snapshots should settle around 672, and oldest_snap should never drift past 7 days back.

Generating a report

sqlplus perfstat/<password>
SQL> @?/rdbms/admin/spreport.sql

It prompts for begin and end snap IDs and produces a text report covering top SQL by buffer gets and elapsed time, wait events, load profile, and instance efficiency ratios — functionally the SE equivalent of an AWR report.

Risks and Things to Watch

Overhead at this frequency is generally low — sub-second, mostly recursive SQL against V$ views — but on a heavily loaded system you may see brief library-cache/shared-pool activity during each snap. Validate this in non-prod first.

Threshold tuning matters on high-transaction systems. Statspack only captures SQL exceeding thresholds defined in STATS$STATSPACK_PARAMETER. On a busy financial workload, consider raising executions_th so the repository doesn't fill up with noise from trivial, high-frequency statements.

Repository growth can surprise you. A runaway job or unusual reporting query can spike SQL diversity and balloon snapshot size fast. Use the weekly tablespace query above as an early warning — don't rely on AUTOEXTEND MAXSIZE alone to "handle it," since that just delays a disk-full event rather than preventing it.

Retention vs. compliance. A hard 7-day purge can conflict with audit retention requirements in regulated environments. If there's any chance you'll need performance evidence beyond 7 days for an incident review, consider spooling spreport.sql output to an archive nightly, rather than treating the live repository as your only record.

Account hygiene. PERFSTAT should never be used for interactive login. Use a strong password, keep it out of the DBA role, and document it as a service account with a rotation policy in your CMDB.

After patching or upgrades, confirm the scheduler jobs are still in a healthy state:

SELECT job_name, state FROM dba_scheduler_jobs WHERE owner = 'PERFSTAT';

Rolling Back

To pause without losing data — useful during a change freeze:

EXEC DBMS_SCHEDULER.DISABLE('PERFSTAT.STATSPACK_SNAPSHOT');
EXEC DBMS_SCHEDULER.DISABLE('PERFSTAT.STATSPACK_PURGE');

To remove Statspack entirely:

sqlplus / as sysdba
SQL> @?/rdbms/admin/spdrop.sql
-- Only after confirming nothing else relies on this tablespace
DROP TABLESPACE perfstat INCLUDING CONTENTS AND DATAFILES;

Final Thoughts

Statspack isn't as polished as AWR, but it's a fully licensed, zero-cost, battle-tested tool that does the job well on Standard Edition. The setup above — Level 5 snapshots every 15 minutes with extended purging at 7 days — gives you solid visibility into SQL and wait-event trends without the licensing risk. Run it through a full 7-day cycle in non-production first to confirm growth rate and purge behavior before promoting it to a mission-critical production instance.


Have you run Statspack on a 19c Standard Edition environment? Drop your tuning thresholds or retention strategy in the comments — always curious how other teams are handling this on regulated, mission-critical systems.