Under the hood

Last updated:

|Edit this page

On this page

There are two types of data sources we are able to connect data from—integrations we've built and custom sources you can link.

Connectors

Connectors use a combination of Temporal and the open source data loading tool, dlt to sync data from your source to PostHog's storage in S3. You can think of this as a complete extract and load tool where you simply provide the credentials and we will take care of the rest of the data flow.

Custom sources

The second option are custom sources. With custom sources, each table in the warehouse is backed by one or more files in your object storage system (i.e. S3, GCS). We never "ingest" your data into our systems. Instead, on every query, we'll read directly from your object storage system. This means you can store and query as much data as you want.

This also means data freshness is determined by you. Data is as up-to-date as you set it up to be.

Questions? Ask Max AI.

It's easier than reading through 609 docs articles.

Community questions

  • Peter    a year ago

    Postgres

    Hello,

    1. The "Under the hood" sections states this:

    We never "ingest" your data into our systems. Instead, on every query, we'll read directly from your object storage system.

    But when I connect a Postgres database, it does seem to ingest all tables instead of querying them on-demand. Is the information outdated?

    1. Is there a fixed IP range the connections will be made from? I'd like to avoid exposing the port to the whole internet.

    2. Do you plan verifying server certificates? Your Postgres connector seems to require TLS but doesn't verify neither CA nor hostname.

    • Matt      9 months ago

      My reading of the docs above is that all the "Connectors" sync data from a source to s3 object storage, including the Postgres one -- so the database will be queried when the sync runs. It's the object storage that is subsequently queried on-demand.

Was this page useful?

Next article

Tutorials and guides

Got a question which isn't answered below? Head to the community forum to let us know! How to set up Stripe reports How to set up Hubspot reports How to set up Zendesk reports How to set up Google Ads reports How to query Supabase in PostHog How to query a CSV in PostHog

Read next article

PostHog.com doesn't use third party cookies - only a single in-house cookie.

No data is sent to a third party.

Ursula von der Leyen, President of the European Commission