What's new
- Compute: Updated PostgreSQL versions to 14.8 and 15.3, respectively.
- Compute: Implemented a
cargo neon
utility to facilitate setting up the Neon project locally. Setup instructions have been updated to reflect this change.
The latest product updates from Neon
RSS feedcargo neon
utility to facilitate setting up the Neon project locally. Setup instructions have been updated to reflect this change.Proxy: Neon uses compute endpoint domain names to route incoming client connections. For example, to connect to the compute endpoint ep-mute-recipe-239816
, we ask that you connect to ep-mute-recipe-239816.us-east-2.aws.neon.tech
. However, the PostgreSQL wire protocol does not transfer the server domain name, so Neon relies on the Server Name Indication (SNI) extension of the TLS protocol to do this. Unfortunately, not all PostgreSQL clients support SNI. When these clients attempt to connect, they receive an error indicating that the "endpoint ID is not specified".
As a workaround, Neon provides a special connection option that allows clients to specify the compute endpoint they are connecting to. The connection option was previously named project
. This option name is now deprecated but remains supported for backward compatibility. The new name for the connection option is endpoint
, which is used as shown in the following example:
postgres://<user>:<password>@ep-mute-recipe-239816.us-east-2.aws.neon.tech/main?options=endpoint%3Dep-mute-recipe-239816
For more information about this special connection option for PostgreSQL clients that do not support SNI, refer to our connection workarounds documentation.
failed to flush page requests
errors by adjusting Pageserver timeout settings.Timed out while waiting for WAL record
errors. The additional information is used for diagnostic purposes.Lsn::INVALID
value was sent from the Safekeeper to the Pageserver if there were no WAL records to send. This incorrectly indicated to the Pageserver that the Safekeeper was lagging behind, causing the Pageserver to connect to a different Safekeeper. Instead of Lsn::INVALID
, the most recent commit_lsn
value is now sent instead.US East (N. Virginia) — aws-us-east-1
region. For more information about Neon's region support, see Regions.ip4r
and pg_hint_plan
extensions. For more information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.lz4
and zstd
WAL compression methods.procps
, which is a set of utilities for process monitoring.syscalls
changes in the WAL redo seccomp
(secure computing mode) code to ensure AArch64 support.disk_size
and instance_type
properties to the Pageserver API. This data is required to support assigning Neon projects to Pageservers based on Pageserver disk usage.proxy_io_bytes_per_client metric
values.wal_backup_lsn
is now advanced after each WAL segment is offloaded to storage to avoid lags in WAL segment cleanup.None
value instead. A non-wildcard common name is now set, and an error is reported if a None
value is passed.libpq
configuration, making it possible to enable authentication for only the management HTTP API or the Compute API.$NEON_AUTH_TOKEN
variable handling when connecting from a compute to Pageservers and Safekeepers.Compute: Released a new pg_tiktoken
PostgreSQL extension, created by the Neon engineering team. The extension is a wrapper for OpenAI’s tokenizer. It provides fast and efficient tokenization of data stored in a PostgreSQL database.
The extension supports two functions:
tiktoken_encode
function takes text input and returns tokenized output, making it easier to analyze and process text data.tiktoken_count
function returns the number of tokens in a text, which is useful for checking text length limits, such as those imposed by OpenAI’s language models.For more information about the pg_tiktoken
extension, refer to the blog post: Announcing pg_tiktoken: A Postgres Extension for Fast BPE Tokenization. The pg_tiktoken
code is available on GitHub.
Compute: Added support for the PostgreSQL prefix
, hll
and plpgsql_check
extensions. For more information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.
Compute, Pageserver, Safekeeper: Added support for RS384 and RS512 JWT tokens, used to securely transmit information as JSON objects.
Pageserver: Removed the block cursor cache, which provided little performance benefit and would hold page references that caused deadlocks.
Autoscaling: Added support for scaling Neon's local file cache size when scaling a virtual machine.
Broken pipe
error on the Pageserver. In case of a failure, the WAL redo process is now restarted, and requests to apply redo records are retried automatically.Failed to open WAL file
warnings that appeared in the Safekeeper log files. The warnings were due to an outdated truncate_lsn
value on the Safekeeper, which caused the walproposer (the Postgres compute node) to download WAL records starting from a Log Sequence Number (LSN) that was older than the backup_lsn
. This resulted in unnecessary WAL record downloads from cold storage.rum
and pgTAP
extensions. For more information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.pg_graphql
pg_jsonschema
pg_hashids
pgrouting
hypopg
autoinc
insert_username
moddatetime
refint
Compute: Added support for the PostgreSQL xml2
and pgjwt
extensions. For more information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.
Compute: Updated the versions for the following PostgreSQL extensions:
address_standardizer
, address_standardizer_data_us
, postgis
, postgis_raster
, postgis_tiger_geocoder
, postgis_topology
extensions to version 3.3.2
.plv8
, plls
, plcoffee
extensions to 3.1.5
.h3_pg
extension to 4.1.2
.Updating an extension version requires running an ALTER EXTENSION <extension_name> UPDATE TO <new_version>
statement. For example, to update the postgis_topology
extension to the newly supported version, run this statement:
ALTER EXTENSION postgis_topology UPDATE TO '3.3.2';
Pageserver: Corrected the storage size metrics calculation to ensure that only active branches are counted.
Proxy: Enabled OpenTelemetry
tracing to capture all incoming requests. This change enables Neon to perform an end-to-end trace when a new
connection is established.
pgvector
, plls
and plcoffee
extensions. For more information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.TCP_NODELAY
protocol option. The TCP_NODELAY
option causes segments to be sent as soon as possible, even if there is only a small amount of data. For more information, refer to the TCP protocol man page.Compute: Added support for the PostgreSQL postgis-sfcgal
extension. For more information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.
Compute: Added support for International Components for Unicode (ICU), which permits defining collation objects that use ICU as the collation provider. For example:
CREATE COLLATION german (provider = icu, locale = 'de');
unit
extension. For more information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.gc
) interval from 100 seconds to 60 minutes. This change reduces the frequency of layer map locks.bloom
pgrowlocks
intagg
pgstattuple
earthdistance
address_standardizer
address_standardizer_data_us
pg8000
Python PostgreSQL driver, version 1.29.3 and higher, now supports connecting to Neon.EXPLAIN
that show prefetch hits and misses for sequential scans.pg_prewarm
PostgreSQL extension, which utilizes the above-mentioned sequential scan prefetch feature. The pg_prewarm
extension provides a convenient way to load data into the PostgreSQL buffer cache after a cold start. For information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.storage_sync
operation to make it more robust and reliable while syncing files between Pageservers and S3.postgres
repository to the neon
repository and created a separate wal_redo
binary in order to reduce the amount of change in the postgres
repository codebase.--daemonize
option from the CLI process that starts the Pageserver and Safekeeper storage components. The required library is no longer being maintained and the option was only used in our test environment.wal_log_hints
parameter, which is the default PostgreSQL setting. The Pageserver-related issue that required enabling wal_log_hints
has been addressed, and enabling wal_log_hints
is no longer necessary.state
field to the TimelineInfo
struct that is returned by the timelines
internal management API endpoint. Timeline state
information improves observability and communication between Pageserver modules.INSERT ... ON CONFLICT
handling issue for speculative Write-Ahead Log (WAL) record inserts. Under certain load conditions, records added with INSERT ... ON CONFLICT
could be replayed incorrectly.compaction_period
setting to 20 seconds to reduce the frequency of polling that is performed to determine if compaction is required. The frequency of polling with the previous setting of 1 could result in excessive CPU consumption when there are numerous tenants and projects.h3_pg
and plv8
PostgreSQL extensions. For information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.backpressure_throttling_time
function that returns the total time spent throttling since the system was started.PostGIS
extension, version 3.3.0. For information about PostgreSQL extensions supported by Neon, see PostgreSQL extensions.options
, application_name
, and replication
connection parameters to compute nodes.uuid-ossp
extension binaries, which provide functions for generating universally unique identifiers (UUIDs). CREATE EXTENSION "uuid-ossp"
is now supported. For information about extensions supported by Neon, see Available PostgreSQL extensions.zenith.page_server_connstring
to neon.pageserver_connstring
zenith.zenith_tenant
to neon.tenant_id
zenith.zenith_timeline
to neon.timeline_id
zenith.max_cluster_size
to neon.max_cluster_size
wal_acceptors
to safekeepers
zenith_admin
role to cloud_admin
.fullbackup
endpoint that works like basebackup but also sends relational files.psql
copy.DROP DATABASE
is executed.CREATE EXTENSION
statement for users that are not database owners.max_lsn_wal_lag
default setting to avoid constant reconnections during normal work.wal_connection_attempts
maintenance, preventing busy reconnection loops.