PostgreSQL Performance Tuning & Operations - Complete Notes

A measurement-first, practical and interview-focused guide to server resources, memory, planner settings, connection pooling, observability, slow-query diagnosis, partitioning, capacity planning, and safe production operations.

00. The measurement-first mental model

Performance tuning is controlled problem solving. Define the user-visible problem, capture a baseline, locate the constrained resource, form one testable hypothesis, change the smallest relevant thing, and compare the same measurements again.

Treat PostgreSQL like a busy restaurant. Buying a larger pantry does not help if orders wait at the payment desk. Adding more tables can make service worse when the kitchen is already full. First measure where time is spent: waiting for a worker, waiting for a lock, reading storage, sorting, writing, or returning rows. Then increase capacity or reduce work at that exact stage.

Layer Question Evidence
User Which operation is slow, how slow, and for whom? p50, p95, p99 latency, errors, timeouts and request traces
Workload Did arrival rate, query mix, data size, or concurrency change? Requests/s, transactions/s, rows/s, deployments and growth
Database Is work executing or waiting? pg_stat_activity, wait events, locks and statement statistics
Plan Does one statement read, spill, loop, or estimate badly? EXPLAIN (ANALYZE, BUFFERS, WAL, SETTINGS)
Host Which physical resource is saturated? CPU, memory, storage latency/queue, filesystem space and network
A setting is not a goal

A higher cache hit ratio, larger shared_buffers, or lower random_page_cost is not automatically success. Success is a workload meeting its latency, throughput, correctness, durability and cost targets with safe operating headroom.

01. Workload, latency, throughput and queues

A useful baseline describes both demand and service. Latency says how long one operation takes; throughput says how many complete per unit time; concurrency says how many are in flight.

  • Latency: report percentiles, not only averages. A 20 ms average can hide a 2 second p99 that harms real users.
  • Throughput: requests, transactions, queries, rows, or bytes completed per second. Name the unit and whether failed work is counted.
  • Concurrency: active requests, active database backends and transactions. More concurrency raises throughput only until a constrained resource saturates.
  • Utilization: the fraction of resource capacity in use. Near saturation, small demand increases can create disproportionate queueing delay.
  • Service time versus wait time: a query can be slow because it is doing work or because it is queued behind a lock, CPU, I/O, connection pool, or rate limit.

Little's Law gives a valuable consistency check for a stable system: concurrency = throughput × average time in system. If 200 requests finish each second and average end-to-end time is 0.05 seconds, about 10 requests are in flight on average. If latency grows while throughput is flat and concurrency rises, work is probably queueing.

Do not compare different workloads

A before-and-after test is meaningful only when data volume, cache state, parameters, query mix, concurrency and observation window are comparable. A faster result after traffic dropped does not prove the configuration change helped.

02. Find the constrained resource

PostgreSQL performance is bounded by CPU, memory, storage, network, locks, connection slots, or the amount of work requested. Tuning the wrong resource often moves no useful metric.

Constraint Common signals First questions
CPU Sustained busy cores, runnable queue, CPU-heavy plans Are rows filtered late, expressions expensive, or concurrency excessive?
Memory Swap, OOM kills, reclaim pressure, unstable latency How many backends and simultaneous memory operations exist?
Storage High read/write latency, queue depth, fsync waits, temp I/O Is the access necessary, sequential/random, cached, or spilled?
Locks wait_event_type = 'Lock', blocker chains Which transaction owns the lock and why is it still open?
Connections Pool wait, connection errors, many mostly idle backends Is server concurrency bounded independently from client count?
Network/client ClientRead/ClientWrite, slow result consumption Are too many or too-wide rows crossing the network?

CPU utilization alone is incomplete. One busy core can limit a serial query while total host CPU looks low. Conversely, 100 percent CPU can be healthy at peak if latency stays within target and headroom is deliberate. Storage IOPS alone is also incomplete: latency and queue depth show whether requests are completing promptly.

03. Shared buffers, OS cache and huge pages

PostgreSQL reads table and index blocks through its shared buffer cache, while the operating system also caches file data. Memory planning must leave room for both caches, backend memory, maintenance, the kernel, monitoring, poolers and other processes.

shared_buffers

shared_buffers is the amount of memory PostgreSQL dedicates to shared database blocks. It is allocated at server start. Current official guidance suggests about 25 percent of RAM as a reasonable starting point on a dedicated server with at least 1 GB, then measurement. Allocating more than roughly 40 percent is unlikely to work better because PostgreSQL also relies on the OS cache. These are starting points, not universal formulas.

  • A buffer hit avoids a filesystem read but still consumes CPU, locking and memory bandwidth.
  • A shared-buffer miss can still be satisfied quickly from the OS page cache.
  • Increasing the setting does not repair a query that reads millions of unnecessary rows or lacks a suitable index.
  • A larger dirty-buffer population can increase checkpoint work. Official guidance notes that a larger value often needs a corresponding max_wal_size increase to spread writes.
  • The database-wide block hit ratio is not a sufficient target. Sequential analytics can have a low ratio and be healthy; a tiny hot OLTP working set can have a high ratio while one query is still bad.

Huge pages

Huge pages map the main shared memory region with fewer page-table entries. This can reduce CPU time spent managing memory mappings, especially with large shared memory. On supported systems, huge_pages = try requests them and falls back if unavailable, on makes failure fatal, and off disables the request. Check SHOW huge_pages_status after startup rather than assuming they were used.

Explicit huge pages and transparent huge pages differ

PostgreSQL's huge_pages setting controls the main shared memory request. Operating system transparent huge pages are a separate mechanism and can have different latency behavior. Follow the PostgreSQL version and operating-system guidance, test on representative load, and verify startup plus memory telemetry.

04. Query and maintenance memory

PostgreSQL memory settings are different budgets with different multiplication rules. work_mem is not a per-session or per-query cap.

work_mem and multiplication

Each sort, hash table, hash aggregate, Memoize cache and similar executor operation can use a memory allowance based on work_mem before temporary-file work. One plan may contain several such nodes. Parallel workers may each build their own state, and many sessions may do this together. Hash operations derive their limit from work_mem × hash_mem_multiplier.

Reason about an upper-risk scenario, not a fake exact total
Potential executor memory
  ≈ active memory nodes
  × processes participating per query
  × concurrent heavy queries
  × applicable memory allowance

Example risk:
  3 sort/hash nodes × 3 processes × 20 queries × 64 MB
  = 11,520 MB before backend overhead and other memory

Nodes do not always peak simultaneously and may use less than their allowance, so this is not an exact reservation formula. It is a safety model showing why a global jump from 4 MB to 256 MB can exhaust a server. Prefer reducing input rows and row width. Then use a scoped experiment for a known reporting job.

Use a transaction-local allowance for a measured workload
BEGIN;
SET LOCAL work_mem = '128MB';

EXPLAIN (ANALYZE, BUFFERS, SETTINGS)
SELECT customer_id, SUM(total)
FROM orders
WHERE ordered_at >= DATE '2026-01-01'
GROUP BY customer_id;

COMMIT;

maintenance_work_mem

maintenance_work_mem is used by operations such as VACUUM, CREATE INDEX and ALTER TABLE ADD FOREIGN KEY. A larger budget can make index creation or vacuum work faster, but concurrent maintenance still multiplies memory. Each autovacuum worker can use memory up to autovacuum_work_mem, or inherit maintenance_work_mem when the former is -1. Budget for the configured and observed worker count.

Temporary files are evidence, not permission to raise memory globally

Inspect which statement and plan node spills, the temporary bytes, concurrent frequency and total host headroom. A spill may be cheaper and safer than keeping every occasional large sort in RAM.

05. Planner cache, I/O costs and statistics

Planner settings describe PostgreSQL's cost model. They do not make hardware faster. Change them only when measurements show the model consistently misrepresents the environment.

Setting Meaning Common mistake
effective_cache_size Planner estimate of cache available to one query Treating it as allocated or reserved memory
effective_io_concurrency Expected concurrent storage I/O and prefetch behavior Setting it high without storage latency testing
seq_page_cost Relative cost of a sequential page fetch, default 1.0 Reading the number as milliseconds
random_page_cost Relative cost of non-sequential page access, default 4.0 Lowering it only to force one desired index plan
default_statistics_target Default detail collected for columns, default 100 Raising every column rather than the skewed one

effective_cache_size should consider shared buffers plus the useful portion of the OS file cache, adjusted for concurrent queries and overlap. It changes estimated index costs but allocates nothing. A higher estimate can make index scans look more attractive.

effective_io_concurrency tells PostgreSQL how much parallel I/O the storage can usefully handle and controls prefetch distance on supported systems. Current releases allow values from 1 to 1000, with 0 disabling asynchronous I/O requests for this purpose. A value that helps a high-IOPS device can overload slower or shared storage, so compare query latency and device queueing. Tablespaces can override relevant I/O and cost settings.

Lowering random_page_cost relative to seq_page_cost favors index access. Equal values can make sense for a fully cached database. A random cost below the sequential cost is not physically sensible. Calibrate with a family of representative queries, not one plan.

Statistics quality before cost tuning

The planner estimates row counts from table size and statistics produced by ANALYZE. Stale statistics, skew, correlated columns and parameter-sensitive distributions can cause wrong scan, join and memory choices. First run or confirm auto-analyze, compare estimated and actual rows, and inspect pg_stats.

Target a skewed predicate column
ALTER TABLE orders
  ALTER COLUMN tenant_id SET STATISTICS 1000;

ANALYZE orders (tenant_id);

-- Return to the database default if the larger target is no longer useful.
ALTER TABLE orders
  ALTER COLUMN tenant_id SET STATISTICS -1;

Larger targets allow larger most-common-value lists and histograms and use a larger sample, at the cost of more analysis time and catalog space. Use extended statistics for cross-column dependencies, multivariate common values, or distinct counts. Increasing an unrelated global target cannot fix missing correlation knowledge by itself.

06. Checkpoints and WAL performance overview

WAL protects durability and checkpoints bound recovery work. Operational tuning aims to avoid bursty writes while preserving the required durability and recovery objectives.

  • checkpoint_timeout bounds time between automatic checkpoints. A longer interval can mean more crash-recovery work.
  • max_wal_size is a soft WAL-size threshold that can trigger checkpoints. It is not a strict disk-space cap.
  • checkpoint_completion_target, default 0.9, spreads checkpoint writes across most of the interval. Lowering it usually creates a sharper I/O burst.
  • Frequent requested checkpoints suggest WAL volume is reaching the threshold or explicit checkpoints are occurring. Compare timed and requested counters, WAL rate and storage latency.
  • wal_compression can reduce full-page-image WAL volume at a CPU cost. Measure both.
  • Relaxing synchronous_commit can reduce commit wait for selected transactions but changes the window in which recently acknowledged transactions can be lost after an OS crash. It does not make the database inconsistent. Use only with an explicit durability decision.
Keep durability decisions separate from ordinary tuning

Never disable fsync or use unsafe storage behavior to win a benchmark on a durable production system. Topic 8 explains WAL, checkpoints and crash recovery in depth. Here the operational rule is to observe WAL rate, checkpoint cause, write/sync time, disk capacity and recovery requirements together.

07. Connection model, backend cost and pooling

PostgreSQL starts a separate server process for each accepted client connection. Connections are useful sessions, but active SQL concurrency must be bounded by what CPU, memory and storage can serve efficiently.

Each backend has process and session state, can allocate private executor memory, participates in shared structures, and adds scheduling work. Idle connections are cheaper than active heavy queries but are not free. Hundreds of clients all becoming active can multiply work_mem, lock contention and CPU run queues at once.

max_connections is a hard server admission limit, not a target concurrency. Raising it also increases some shared resource allocations and can convert a clean connection rejection into host-wide memory pressure and timeouts. Keep emergency slots through reserved_connections and superuser_reserved_connections as appropriate, and test the actual administrative path.

Separate client concurrency from database concurrency

An application can support thousands of connected users while allowing only a measured number of concurrent PostgreSQL server connections. A pool absorbs short bursts into a bounded queue. Queueing at a visible pool is safer than letting every request consume backend memory and compete inside the database.

  • Size pools across all application instances, jobs, consoles and replicas, not per instance.
  • Reserve capacity for migrations, monitoring, autovacuum and emergency administration.
  • Use finite pool-acquisition and statement timeouts aligned with the request deadline.
  • Monitor pool wait time and queue length. A pool can protect PostgreSQL while exposing demand.
  • Avoid one large pool per tenant or role without calculating the total server connections.

08. PgBouncer modes and compatibility

PgBouncer is a lightweight connection pooler. Its pool mode decides when a PostgreSQL server connection may be reassigned to another client, which also decides which session features remain safe.

Mode Server connection released Tradeoff
Session When the client disconnects Best compatibility, least multiplexing for long-lived clients
Transaction When the transaction finishes Strong multiplexing; application must not depend on session state
Statement When one statement finishes Most restrictive; multi-statement transactions are disallowed

In transaction pooling, consecutive transactions from one client can run on different server backends. Session-level SET, LISTEN, session advisory locks, holdable cursors, and temporary tables that preserve rows cannot be assumed to follow the client. Transaction-local state such as SET LOCAL remains tied to its transaction.

PgBouncer can support protocol-level named prepared statements in transaction and statement modes when max_prepared_statements is nonzero. SQL-level PREPARE, EXECUTE and DEALLOCATE do not receive that tracking and remain incompatible with transaction pooling. Confirm client driver behavior and the deployed PgBouncer version. DDL that changes prepared result types may require a planned RECONNECT.

Transaction pooling changes session semantics by design

Do not select it only because it supports more clients. Inventory application features, test migrations and drivers, set application_name, verify authentication and TLS, and observe SHOW POOLS plus SHOW STATS. Session pooling still avoids repeated PostgreSQL connection setup, but does not reduce steady server connections for long-lived clients.

09. Monitoring semantics and safe observation

PostgreSQL exposes current activity and cumulative counters. Know whether a value is a live state, a counter since reset, a transaction-scoped snapshot, or an optional timing metric before drawing conclusions.

  • pg_stat_activity is current per-process state. It is not a historical query log.
  • Cumulative views such as pg_stat_database describe totals since stats_reset. Compare rates over equal intervals.
  • Statistics fetched inside a transaction can be cached for consistency. Run monitoring queries in short transactions so repeated reads can see fresh values.
  • I/O timing columns remain zero unless track_io_timing is enabled, and mixed periods before and after enabling it require caution.
  • Access is permission-sensitive. Use pg_read_all_stats for an appropriate monitoring role instead of making tools superusers.
  • Resetting statistics removes the baseline. Record the reset time and preserve monitoring data externally before intentional resets.

10. Activity, locks and wait events

Start an incident by separating active CPU/I/O work, lock waits, client waits and idle transactions. The state and wait_event columns answer different questions.

Find long-running or waiting sessions
SELECT pid,
       usename,
       application_name,
       client_addr,
       state,
       wait_event_type,
       wait_event,
       now() - query_start AS query_age,
       now() - xact_start AS transaction_age,
       backend_xid,
       backend_xmin,
       left(query, 160) AS query
FROM pg_stat_activity
WHERE pid <> pg_backend_pid()
ORDER BY query_start NULLS LAST;

state = 'active' means the backend is executing a query, but a non-null wait event means it is currently waiting somewhere. idle in transaction is operationally dangerous because it can retain locks and an old snapshot even though no statement is executing. ClientRead usually means PostgreSQL is waiting for the client to send more work, not that storage reads are slow.

Link blocked sessions to blockers
SELECT blocked.pid AS blocked_pid,
       blocked.wait_event_type,
       blocked.wait_event,
       blocker.pid AS blocker_pid,
       blocker.state AS blocker_state,
       now() - blocker.xact_start AS blocker_xact_age,
       left(blocked.query, 120) AS blocked_query,
       left(blocker.query, 120) AS blocker_query
FROM pg_stat_activity AS blocked
CROSS JOIN LATERAL
     unnest(pg_blocking_pids(blocked.pid)) AS b(blocker_pid)
JOIN pg_stat_activity AS blocker
  ON blocker.pid = b.blocker_pid;

pg_locks provides lockable-object detail and granted status, while pg_blocking_pids() handles blocker relationships, including wait-queue effects, more reliably than a simplistic self-join. Termination is a last step: identify the owner, business effect, transaction and rollback cost. Prefer fixing transaction scope, stable lock order and timeouts.

11. Database, I/O and checkpoint statistics

Cumulative views answer cluster and database trend questions. Convert counters into interval rates and pair them with workload volume.

View Useful signals Interpretation caution
pg_stat_database commits, rollbacks, blocks, tuples, temp bytes, deadlocks, session time Database-wide totals can hide one expensive statement
pg_stat_io reads, writes, extends, evictions, reuse and timing by backend/object/context Cluster-wide and version-dependent; not every I/O path is represented
pg_stat_bgwriter background-writer cleaning, limits and buffer allocation Current releases separate checkpoint work
pg_stat_checkpointer timed/requested checkpoints, write/sync time and buffers Use only where the server version provides this view
Database workload and spill overview
SELECT datname,
       xact_commit,
       xact_rollback,
       blks_read,
       blks_hit,
       temp_files,
       pg_size_pretty(temp_bytes) AS temp_written,
       deadlocks,
       stats_reset
FROM pg_stat_database
WHERE datname IS NOT NULL
ORDER BY temp_bytes DESC;

A block hit percentage from blks_hit / (blks_hit + blks_read) describes PostgreSQL shared-buffer activity, not the OS cache and not physical-device reads. A high percentage does not prove queries are efficient. Use pg_stat_io to distinguish backend types and I/O contexts, and use host telemetry for device-level latency and saturation.

Version-aware checkpoint views

PostgreSQL 17 and later expose pg_stat_checkpointer separately. In PostgreSQL 16 and earlier, checkpoint counters such as requested/timed checkpoints and checkpoint write/sync time are in pg_stat_bgwriter. Monitoring SQL and dashboards must be matched to the server's major version.

12. Finding expensive SQL with pg_stat_statements

pg_stat_statements groups structurally equivalent statements and accumulates planning and execution metrics. It helps rank total workload cost, average cost, calls, I/O and variability.

One-time server and database setup
# postgresql.conf, then restart because shared memory is required
shared_preload_libraries = 'pg_stat_statements'
compute_query_id = on

-- Run in every database where the view should be available.
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

The library tracks across the cluster, while the SQL extension objects are installed per database. Entries are keyed by database, user, query identifier and top-level status. Literal constants are normalized, so a representative query text can contain placeholders. queryid is valuable for joining telemetry but is not guaranteed stable across PostgreSQL major versions.

Rank statements by total execution demand
SELECT queryid,
       calls,
       round(total_exec_time::numeric, 1) AS total_exec_ms,
       round(mean_exec_time::numeric, 2) AS mean_exec_ms,
       round((total_exec_time / NULLIF(sum(total_exec_time) OVER (), 0)
              * 100)::numeric, 2) AS exec_share_pct,
       rows,
       shared_blks_read,
       temp_blks_written,
       left(query, 180) AS sample_query
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 20;
  • Total time: best starting point for overall capacity savings.
  • Mean time: finds individually slow calls but can hide tail latency.
  • Calls: a tiny query can dominate CPU through extreme frequency.
  • Rows: relate returned/affected rows to calls and application expectations.
  • Shared blocks: reveal cache and read demand; timing needs I/O timing enabled.
  • Temporary blocks: reveal spill volume by statement family.
  • WAL bytes: reveal write amplification for modifying statements.
  • Standard deviation: large variation suggests skew, cache, locks, plans or parameter-dependent behavior needing traces or logs.

Planning columns remain zero unless pg_stat_statements.track_planning is enabled. That option can add noticeable contention for frequently planned identical queries, so enable it after evaluating overhead. Track reset time and query-entry deallocations from pg_stat_statements_info. Reset only with a deliberate measurement window.

13. A repeatable slow-query workflow

Query tuning starts with the exact workload context and ends with production verification. It is not a contest to make EXPLAIN display an index.

  1. Define impact: endpoint/job, time window, p95/p99, timeout rate, users, and SLO.
  2. Check waits: determine whether time is in a pool queue, lock, client, I/O, or execution.
  3. Identify the statement family: use trace context, logs, query ID and pg_stat_statements.
  4. Reproduce context: representative parameters, role, search path, prepared-plan behavior, data volume, statistics and relevant settings.
  5. Capture the plan safely: start with EXPLAIN; use EXPLAIN (ANALYZE, BUFFERS, WAL, SETTINGS) only when execution is safe.
  6. Find the first cause: earliest major estimate error, excessive rows/width, repeated loops, bad access path, spill, or lock.
  7. Change one cause: SQL, index, statistics, data model, transaction scope, or a scoped configuration.
  8. Compare: result correctness, plan, buffers, temp I/O, WAL, latency under concurrency, CPU and storage.
  9. Roll out and watch: canary where possible, preserve rollback, and inspect fleet and tail metrics.
ANALYZE executes the statement

EXPLAIN ANALYZE on an INSERT, UPDATE, DELETE or MERGE performs the write, fires triggers, waits for locks and generates WAL. A transaction plus rollback can undo transactional database changes, but not every external side effect.

14. Slow logging and auto_explain

Statistics find aggregate offenders. Logs and traces preserve individual occurrences, parameters, timing context and errors. Capture enough evidence without leaking data or overwhelming storage.

  • log_min_duration_statement = '250ms' logs every completed statement at or above the threshold. The default -1 disables it; zero logs every duration.
  • log_min_duration_sample with log_statement_sample_rate samples qualifying traffic when logging every query would be too expensive.
  • Include timestamp, process/session identity, database, user and application in log_line_prefix. Use structured CSV or JSON logs when the pipeline supports them.
  • Parameter values and SQL can contain secrets or personal data. Apply access control, retention, redaction and a deliberate log_parameter_max_length_on_error policy.
  • log_temp_files identifies large temporary files. Select a threshold that is useful at production volume.

auto_explain can log plans for statements crossing auto_explain.log_min_duration. Load it only into intended sessions or through a planned preload setting. auto_explain.log_analyze = on collects actual execution detail, but per-node timing can affect every statement even when it is not eventually logged. Sampling and auto_explain.log_timing = off can reduce overhead. Test overhead and log volume before production rollout.

15. Partitioning mental model and syntax

A partitioned table is a logical parent whose rows are routed into physical child tables according to partition bounds. Partitioning is primarily a data-management and pruning tool, not a universal substitute for indexes.

Method Rule Typical use
Range Non-overlapping lower-inclusive, upper-exclusive ranges Time-series retention and ordered key ranges
List Explicit key values per partition Small stable sets such as region or business category
Hash Modulus and remainder distribute keys Even spread when natural range/list grouping is unsuitable
Default Rows not matching another partition Operational safety net, monitored for unexpected routing
Monthly range partitions with a default safety net
CREATE TABLE events (
  event_id bigint GENERATED ALWAYS AS IDENTITY,
  occurred_at timestamptz NOT NULL,
  tenant_id bigint NOT NULL,
  payload jsonb NOT NULL,
  PRIMARY KEY (occurred_at, event_id)
) PARTITION BY RANGE (occurred_at);

CREATE TABLE events_2026_07 PARTITION OF events
FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');

CREATE TABLE events_2026_08 PARTITION OF events
FOR VALUES FROM ('2026-08-01') TO ('2026-09-01');

CREATE TABLE events_default PARTITION OF events DEFAULT;
List and hash forms
CREATE TABLE invoices (
  region text NOT NULL,
  invoice_id bigint NOT NULL,
  amount numeric(12,2) NOT NULL
) PARTITION BY LIST (region);

CREATE TABLE invoices_apac PARTITION OF invoices
FOR VALUES IN ('IN', 'SG', 'JP');

CREATE TABLE sessions (
  tenant_id bigint NOT NULL,
  session_id uuid NOT NULL,
  data jsonb
) PARTITION BY HASH (tenant_id);

CREATE TABLE sessions_h0 PARTITION OF sessions
FOR VALUES WITH (MODULUS 4, REMAINDER 0);
-- Create remainders 1, 2 and 3 as well.

Inserts through the parent are routed to a matching leaf. If no partition matches and no default exists, the insert fails. Updating a partition key can move a row between partitions and may produce serialization failures when concurrent sessions see the movement.

16. Pruning, indexes and constraints

Partition pruning removes child plans whose bounds cannot match the predicate. It is driven by partition bounds, not by indexes, and can occur both while planning and during execution.

A predicate aligned with the partition key
EXPLAIN (ANALYZE, BUFFERS)
SELECT tenant_id, count(*)
FROM events
WHERE occurred_at >= TIMESTAMPTZ '2026-07-10 00:00:00+00'
  AND occurred_at <  TIMESTAMPTZ '2026-07-11 00:00:00+00'
GROUP BY tenant_id;

A prepared parameter, subquery result, or nested-loop parameter may become known only during execution, so PostgreSQL can prune at plan initialization or repeatedly during execution. EXPLAIN may report Subplans Removed; execution-time pruning can appear as (never executed) or different loop counts.

  • Functions or casts around the key can prevent the planner from proving the bound. Prefer direct, type-correct range predicates where semantics allow.
  • An index on the partition key is not required for pruning. Add per-partition indexes for selective access inside partitions.
  • Creating an index on the parent creates matching indexes on existing partitions and arranges them for future partitions.
  • Parent CREATE INDEX does not support CONCURRENTLY. For a busy hierarchy, create an invalid index on ONLY the parent, build child indexes concurrently, then attach them.
  • A primary key or unique constraint on a partitioned table must include all partition-key columns and use columns, not expressions, because PostgreSQL cannot enforce cross-partition uniqueness with independent child indexes otherwise.
  • Foreign keys and triggers need explicit operational testing across the hierarchy.

17. Partition maintenance and lifecycle

The major operational benefit of partitioning is treating a bounded slice as a table: prepare it, load it, index it, attach it, detach it, archive it, or drop it.

  1. Create future partitions before the first possible row arrives.
  2. Apply owners, privileges, indexes, storage parameters and monitoring consistently.
  3. Alert on rows entering a default partition and move them deliberately.
  4. Analyze a newly loaded partition before serving plan-sensitive reads.
  5. Test retention detach/drop locks and dependent objects before the production window.
  6. Keep a catalog/runbook of bounds so gaps and overlaps are reviewed before deployment.

Attaching an existing table normally requires PostgreSQL to scan it while holding a strong lock to verify the proposed bound. Add a matching validated CHECK constraint first so the scan can often be avoided, then remove the redundant constraint after attachment if appropriate. If a default partition exists, add a constraint proving it excludes the new range; otherwise the default partition may be scanned under ACCESS EXCLUSIVE lock.

Prepare and attach a preloaded partition
CREATE TABLE events_2026_09 (LIKE events INCLUDING DEFAULTS
                                        INCLUDING CONSTRAINTS
                                        INCLUDING STORAGE);

ALTER TABLE events_2026_09
  ADD CONSTRAINT events_2026_09_bound
  CHECK (occurred_at >= TIMESTAMPTZ '2026-09-01 00:00:00+00'
     AND occurred_at <  TIMESTAMPTZ '2026-10-01 00:00:00+00');

-- Load, validate, index and analyze before the short attach operation.
ALTER TABLE events ATTACH PARTITION events_2026_09
FOR VALUES FROM ('2026-09-01') TO ('2026-10-01');

DETACH PARTITION CONCURRENTLY, where its restrictions permit, reduces the parent lock level but uses multiple transactions and cannot run inside a transaction block. Version and hierarchy restrictions matter, so test the exact command. Detaching or dropping a whole old partition avoids row-by-row delete work and the vacuum burden that follows a massive DELETE.

18. When partitioning helps or hurts

Partitioning helps when Partitioning hurts when
Most queries restrict a key that permits strong pruning Queries usually touch every partition
Retention removes complete time/key ranges There is no bounded lifecycle unit
Hot and cold slices need different storage or maintenance All slices have identical access and maintenance
One partition's indexes fit cache better A normal index already solves selective access
Bulk load/attach enables controlled ingestion Added routing, DDL and monitoring complexity exceeds benefit
The partition count stays deliberate and bounded One partition per small tenant creates thousands of children

Too many surviving partitions increase planning time and memory. PostgreSQL can handle thousands well when typical queries prune to a small subset, but that is not permission to create unlimited children. Model future tenant count, retention periods and subpartition growth. Benchmark inserts, point queries, broad reports, prepared statements and DDL at the expected partition count.

Partitioning is not sharding

Native partitions remain in one PostgreSQL cluster and share its CPU, memory, WAL, connection limit and failure domain. Partitioning organizes local data. Sharding distributes data and responsibility across servers, adding a different set of consistency and operational problems.

19. Capacity planning

Capacity planning connects business growth to resource demand and preserves headroom for bursts, maintenance, failover and degraded operation.

  1. Forecast data rows, table/index bytes, WAL bytes, backups and logs over the planning horizon.
  2. Measure peak and sustained transactions/s, query mix, concurrent active sessions and result volume, not only daily averages.
  3. Record CPU seconds, read/write bytes, IOPS, WAL and temporary bytes per business unit such as 1,000 orders.
  4. Model memory as shared allocations plus backend/session memory, query operations, autovacuum, maintenance, OS cache, kernel and safety margin.
  5. Model storage for primary data, indexes, bloat allowance, pg_wal, temporary files, snapshots/backups and growth during maintenance.
  6. Load test at future data size and realistic skew. Small synthetic tables produce misleading plans and cache behavior.
  7. Test the degraded case: replica unavailable, backup running, autovacuum active, cache cold, or one storage path slower.

Alert on predicted exhaustion, not only current failure. Disk-space runway in days, WAL generation rate, connection-pool saturation, oldest transaction, replica lag, vacuum backlog and CPU headroom are leading indicators. Capacity also includes recovery time: a backup that cannot be restored within the objective is not adequate capacity.

20. OS, storage and network observations

PostgreSQL statistics explain database behavior; operating-system metrics confirm what the hardware actually did. Correlate them on the same timeline.

  • CPU: per-core utilization, run queue, context switches and steal time in virtual environments. Separate user, system and I/O wait time.
  • Memory: available memory, page faults, reclaim, swap activity and OOM events. Swap-in during database load is a serious latency warning.
  • Storage: read/write latency percentiles, operations/s, bandwidth, queue depth, flush latency and device errors. Confirm filesystem and volume saturation.
  • Space: data filesystem, WAL filesystem, temporary space, inode use and snapshot amplification. Keep emergency headroom and automatic growth alerts.
  • Network: throughput, retransmits, connection establishment and cross-zone latency. Large result sets can make the client or network the bottleneck.

Storage durability claims must cover power-loss behavior, write caches, barriers and the complete stack. A fast benchmark does not prove durable fsync. For managed services, combine provider metrics with PostgreSQL views and document which knobs are unavailable.

21. Safe tuning rollout and runbooks

A performance change is a production change. It needs a hypothesis, owner, validation, rollback, observation window and clear stop conditions.

Runbook field Example
Problem and SLO Checkout p99 is 1.8 s; target is below 750 ms
Evidence window Peak 18:00-19:00 IST, query ID, waits and plan attached
Hypothesis Wrong tenant estimate selects repeated inner scans
Change Column statistics target 500 plus ANALYZE
Success p99 below 750 ms, CPU no higher, same results
Stop/rollback Errors, CPU +15%, new plan regression; restore target and analyze
Owner and duration DB on-call, canary 30 minutes, full watch 24 hours
  1. Capture configuration, plan and dashboard screenshots or exports before change.
  2. Confirm whether the setting reloads, needs a new session, or needs restart.
  3. Test syntax and startup in a non-production environment using the same major version.
  4. Change one variable or one coherent change set with a known rollback.
  5. Canary a role, session, replica, instance or traffic slice when semantics allow.
  6. Compare both target queries and guardrails such as CPU, memory, WAL, locks and other endpoints.
  7. Record the decision and remove temporary settings or instrumentation after the window.

Some settings apply at server start, some after reload, and some per session. Use pg_settings.context, pending_restart, source and sourcefile to verify the effective value and its origin. Do not assume editing a file changed the running server.

22. Common mistakes and edge cases

Mistake Why it fails Better approach
Raise every memory setting Per-node, worker and session multiplication can exhaust RAM Inventory concurrency and test scoped budgets
Set effective_cache_size as RAM allocation It reserves nothing and only changes planner estimates Estimate useful shared plus OS cache for the workload
Raise max_connections for pool exhaustion Moves queueing into PostgreSQL and raises resource pressure Bound server pools and fix leaks/long transactions
Optimize only the slowest single query A frequent 2 ms query may consume more total CPU Rank total time, mean, calls and tail impact
Read cumulative counters as instantaneous Totals depend on uptime/reset length and workload volume Calculate interval deltas and rates
Interpret every wait as a problem Idle processes normally wait; event meaning depends on state Filter by impact, duration, state and workload
Force index scans with cost settings May regress many other queries and hide wrong estimates Fix statistics, SQL and access paths first
Partition every large table Size alone does not guarantee pruning or lifecycle benefit Require an aligned access or retention pattern
Assume transaction pooling preserves sessions The next transaction may use another backend Inventory session features and test the driver
Reset statistics during an incident Destroys the history needed for diagnosis Export first and reset only for a planned window
  • A connection can be active and waiting, so state alone does not prove CPU use.
  • Session statistics and settings can behave differently through transaction pooling.
  • Statistics may lag or reset after restart depending on the facility and configuration.
  • Generic prepared plans can hide tenant skew; compare representative custom and generic plans.
  • A partition default can silently grow forever if no alert and migration process exists.
  • Failover capacity must handle the surviving topology, not only normal balanced traffic.

23. Practice questions

1. A host has 32 GB RAM, 150 active backends, work_mem = 64MB, and plans can have four memory nodes. Why is 150 × 64 MB not a safe estimate?

The allowance is per operation, not per backend. Several nodes, parallel processes and hash multipliers can overlap. Add shared buffers, backend overhead, maintenance, OS cache and safety margin. Measure actual concurrency and use scoped memory for known heavy jobs.

2. The database hit ratio is 99.8 percent but CPU is saturated. What next?

A hit still performs work. Rank statements by total execution time and calls, inspect rows and plans, check per-core CPU and active concurrency, and reduce unnecessary computation or query frequency. Do not enlarge the cache based on this signal.

3. p99 latency rose, throughput stayed flat and active requests doubled. What pattern does this suggest?

Queueing. Use pool wait, database wait events, locks, CPU run queue and storage latency to locate the queue. More concurrency may worsen it.

4. pg_stat_activity shows state = active and wait_event_type = Lock. Is it using CPU?

It is an active statement currently waiting for a lock. Find blockers with pg_blocking_pids(), inspect transaction age and fix the blocking transaction rather than treating the statement as CPU-bound.

5. When is transaction pooling incompatible with an application?

When correctness depends on server-session state across transactions, such as arbitrary SET, LISTEN, session advisory locks, holdable cursors, or persistent temporary data. Inventory protocol-level prepared statement support separately from SQL PREPARE.

6. A monthly partitioned table still scans all partitions for WHERE date(occurred_at) = $1. What should you test?

Test a type-correct half-open predicate directly on occurred_at. The expression may prevent bounds from proving which partitions are impossible. Confirm pruning with EXPLAIN and keep semantics such as time zone explicit.

7. Requested checkpoints greatly outnumber timed checkpoints. What does it suggest?

Checkpoints are being requested before the normal timeout, commonly because WAL reaches its soft size threshold or explicit requests occur. Compare WAL generation, max_wal_size, checkpoint write/sync time, storage headroom and recovery requirements.

8. Why might raising a skewed column's statistics target help?

It can enlarge the sample, common-value list and histogram, improving selectivity and row estimates. Verify the estimate error first, target only useful columns, run ANALYZE and compare plans plus analysis overhead.

9. How can a default partition make attaching the next monthly partition disruptive?

PostgreSQL may scan the default partition under an ACCESS EXCLUSIVE lock to verify it contains no rows for the new range. Keep the default small and add a constraint proving it excludes the new bound before attachment.

10. Why is pg_stat_statements mean time insufficient for tail latency?

It aggregates executions and does not preserve each occurrence. Variability, skew, locks and rare plans can hide behind the mean. Combine standard deviation with traces, slow logs and representative plans.

24. Interview Q&A

Q: How do you approach PostgreSQL performance tuning?

I define the user impact and baseline, compare demand with latency and throughput, classify execution versus waits, rank statement demand, inspect a representative plan, change one root cause, and validate under realistic concurrency with rollback and guardrails.

Q: What is the difference between shared_buffers and effective_cache_size?

shared_buffers allocates PostgreSQL's shared block cache at startup. effective_cache_size allocates nothing; it tells the planner how much shared plus OS cache it can reasonably expect for cost estimation.

Q: Why can work_mem cause an out-of-memory event?

It is a base allowance per sort, hash and similar operation. Multiple nodes, parallel workers and concurrent sessions multiply it, while hashes can use hash_mem_multiplier.

Q: Why not set max_connections very high?

PostgreSQL uses a process per connection and sizes some shared resources from the limit. Too many active backends multiply memory and scheduling contention. A bounded pool keeps server concurrency near measured capacity while allowing more clients to wait safely.

Q: Session versus transaction pooling?

Session pooling assigns a server connection until client disconnect and preserves normal session semantics. Transaction pooling releases it at transaction end for greater multiplexing, but state that must survive across transactions is unsafe unless explicitly supported.

Q: What does a PostgreSQL wait event tell you?

It tells what a backend is waiting on at that observation instant. Interpret it with state, duration, query, transaction, blocker and workload impact. A normal idle backend also waits, so the existence of an event is not itself a problem.

Q: How do you use pg_stat_statements?

I record the measurement window and rank normalized statement families by total execution time, then inspect calls, mean, variability, rows, blocks, temporary I/O and WAL. I connect a query ID to traces or logs and obtain a safe representative plan.

Q: What is partition pruning?

The planner or executor proves from partition bounds and query predicates that some partitions cannot contain matching rows, so those child plans are omitted. Indexes are separate and help selective access inside partitions.

Q: When does table partitioning improve performance?

When common predicates prune to a small subset, indexes become more local, or operations can manage complete slices through attach, detach or drop. It can hurt broad queries and planning when too many partitions remain.

Q: What is your safest method for changing a planner cost?

First correct statistics and access paths. If hardware behavior still differs consistently, test a session-scoped value across a representative query suite and concurrency, record plans and resource metrics, then canary with a rollback rather than forcing one preferred plan.

25. One-screen cheat sheet

  • Goal: meet latency, throughput, correctness, durability and cost targets.
  • Method: baseline, constrain, hypothesize, one change, compare, monitor.
  • Latency: use p50/p95/p99; averages hide tails.
  • Queueing: flat throughput plus rising latency/concurrency is a warning.
  • shared_buffers: PostgreSQL block cache; leave room for OS and processes.
  • effective_cache_size: planner estimate only; allocates no memory.
  • work_mem: per operation, multiplied by nodes/workers/sessions.
  • maintenance_work_mem: maintenance budget; account for worker concurrency.
  • huge pages: verify huge_pages_status after startup.
  • I/O costs: relative planner units, not device milliseconds.
  • Statistics: fix stale/skew/correlation estimates before forcing plans.
  • Checkpoints: compare timed/requested, WAL rate and write/sync time.
  • Connections: process per client; active concurrency is the expensive part.
  • Pooling: bound server connections; monitor pool wait and queue.
  • PgBouncer: transaction mode forbids relying on persistent session state.
  • Activity: combine state, wait event, query age and transaction age.
  • Locks: use pg_blocking_pids() plus activity and lock detail.
  • Counters: calculate deltas/rates and preserve stats_reset.
  • pg_stat_statements: total time, calls, mean, variability, I/O and WAL.
  • Logging: threshold/sample safely; protect parameter data.
  • Partitioning: require aligned pruning or lifecycle benefit.
  • Pruning: driven by bounds, not indexes; can happen during execution.
  • Default partition: monitor it and constrain it before new attachments.
  • Capacity: test future size, peak mix, maintenance and degraded operation.
  • Rollout: effective setting, canary, guardrails, stop condition and rollback.

26. Official reference map

Last reviewed · July 2026 · part of knowledge-base