Skip to main content

​
Introduction

Tyk supports both MongoDB and PostgreSQL as the main database for storing API definitions, policies, user data, and analytics. Please consult the data storage configuration guide for further information relating to how to configure Tyk’s data storage across different database engines.

​
MongoDB

​
Supported Versions

MongoDB is our default storage option. We support the following versions:
  • MongoDB 5.0.x, 6.0.x, 7.0.x (with mongo-go driver).
Note: mongo-go driver has been available since Tyk 5.0.2 and is the default from Tyk 5.3.0.

MongoDB 3.x to 4.4.xPrior to Tyk 5.0.2, Tyk used the mgo driver which supported MongoDB 3.x to 4.4.x, but we no longer test MongoDB versions prior to 5.0 since they are EOL.
We can not guarantee full compatibility with these versions of MongoDB for Tyk and recommend upgrading to a supported MongoDB version. In particular, when using Tyk OAS APIs with Tyk 5.3.0 onwards, the minimum supported version of MongoDB is 5.0.
You can also use the following as a drop-in replacement for MongoDB:

​
Choose a MongoDB driver

From Tyk 5.0.2, we added an option to use the official MongoDB Go driver to connect to MongoDB. We recommend using the mongo-go driver if you are using MongoDB 4.4.x+. For MongoDB versions prior to 4.4, please use the mgo driver. With the mongo-go driver, we support the latest versions of MongoDB (5.0.x, v6.0.x, and v7.0.x) and also features such as the “+srv” connection string and SCRAM-SHA-256. For more details, visit the MongoDB doc: You can configure which driver to use with the MongoDB driver option: Split out your DB This is a no-brainer, but keep Redis and MongoDB off the system running the Gateway, they both use lots of RAM, and with Redis and the Gateway constantly communicating you will be facing resource contention on the CPU for a marginal decrease in latency. So in our setup, we recommend that Redis and MongoDB/PostgreSQL live on their own systems, separate from your Tyk Gateway. If you like, run them together on the same box, that’s up to you. The network topology we like to use is:
  • Two or more Tyk Gateway nodes (load balanced, each Gateway installed on separate machines).
  • A separate MongoDB or PostgreSQL cluster
  • A separate Redis server with fail-over or cluster
  • One Tyk Dashboard node installed on a separate machine
  • One Tyk Pump node installed on a separate machine that handles data transitions
Special notes for DocumentDB
If you are using DocumentDB, capped collections are not supported. See here for more details.
Special notes for MongoDB Atlas To integrate with MongoDB Atlas, make sure the IP firewall connections are whitelisted on the Atlas side, and then use the following Tyk Dashboard configurations to connect: 
- TYK_DB_MONGOURL=mongodb://admin:password@tykdb-shard-00-00.h42pp.mongodb.net:27017,tykdb-shard-00-01.h42pp.mongodb.net:27017,tykdb-shard-00-02.h42pp.mongodb.net:27017/tyk_analytics?authSource=admin - TYK_DB_ENABLECLUSTER=false - TYK_DB_MONGOUSESSL=true
More information on these configuration variables here.

​
MongoDB Sizing Guidelines

The aggregate record size depends on the number of APIs and Keys you have. Each counter size is ~50b, and every aggregated value has its own counter. So an hourly aggregate record is computed like this: 50 * active_apis + 50 * api_versions + 50 * active_api_keys + 50 * oauth_keys, etc. The average aggregate record size (created hourly) on our cloud is about ~ 40KB (a single record includes all the aggregate stats mentioned above). So for 1 million requests per day, it will generate 1KB * 1M request stats (1GB) + 24 * 40KB aggregate stats (~1MB). Per month: 30GB request logs + 30MB aggregate logs MongoDB Working Data Working data in terms of MongoDB is the data you query most often. The graphs displayed on the Tyk Dashboard, except for the Log browser, use aggregated data. So if you rely only on this kind of analytic data, you will not experience issues with working data and memory issues. It is literally hundreds of MBs. Even if you use the Log browser, its usage access is usually quite random, and it is unlikely that you check requests for every request. So it can’t be called working data. And it is ok to store it on disk and allow MongoDB to do the disk lookups to fetch the data. Note, that in order to do fast queries, even from the disk, MongoDB uses indexes. MongoDB recommends that indexes should fit into memory, and be considered working data, but only the part of the index which is commonly used. For example the last month of data. For an aggregate collection, the average index size is 6% of the overall collection. For requests stats, it is around 30%. MongoDB Sizing Example If you serve 1 million requests per day, and require fast access to the last seven days of request logs (usually way less, and the performance of the log viewer is not a concern), with 3 months of aggregated logs, the memory requirements for MongoDB can be as follows: Request_logs_index ( 30% * (1GB * 7) ) + aggregated(3month * 30MB) ~= 2.1GB + 90MB = ~ 2.2GB In addition to storing working data in memory, MongoDB also requires space for some internal data structures. In general, multiplying the resulting number by 2x should be enough. In the above example, your MongoDB server should have around 4.4GB of available memory. Audit Log storage From Tyk Dashboard v5.7+,the audit log can be configured to be stored in the database. If you choose to store the audit logs in the database, you need to account for additional storage for audit logs in the database setup. The size of this table will depend on the number of operations recorded, with each record averaging 1350 to 1450 bytes. Audit Log Considerations
  • Data Generation: The total size of the audit log table will depend on the number of API operations, administrative actions, and system events that are being logged.
  • Daily Estimate: For example, logging 100,000 operations per day results in 135MB to 145MB of additional data daily.
  • Storage Growth: Over time, this can significantly impact your storage requirements, especially in high-traffic environments or systems with comprehensive logging enabled.
Recommendations for Housekeeping the Audit Log Table
  1. Implement Data Retention Policies: Define a clear retention period based on business and regulatory requirements, such as 30, 90, or 180 days. Remove older logs that exceed the retention policy to prevent excessive storage growth.
  2. Archive Older Logs: For long-term storage or compliance purposes, move older logs to external systems such as a data lake, object storage (e.g., S3), or a data warehouse.
  3. Monitor Growth Trends: Use monitoring tools to track the size and growth rate of the audit log table. Adjust retention policies or resources proactively based on observed trends.
  4. Plan for Resource Scaling: Audit log storage can significantly impact overall database size, especially in high-traffic environments. Plan for storage and resource scaling based on daily log growth estimates.
Example Calculation:
  • Daily Logs: 100,000 operations/day
  • Average Record Size: 1400 bytes
  • Storage Growth: 100,000 × 1400 bytes/day = 140MB/day
For 90 days: 140MB × 90 = ~12.6GB MongoDB Database Storage Calculator You can calculate your MongoDB storage requirements by entering your known values in the middle section of the calculator settings below:

​
Database Storage Calculator

Use this calculator to estimate the storage requirements for your database when using Tyk. This tool helps you plan your database infrastructure by calculating storage needs based on your API traffic, analytics retention, and configuration data. The calculator considers factors like requests per second, analytics time-to-live, and the number of APIs and policies you manage. Results show the total storage requirement accounting for your specified utilization threshold.

​
PostgreSQL

How you configure your PostgreSQL installation depends on whether you are installing Tyk from fresh using PostgreSQL, or are migrating from an existing MongoDB instance. Supported Versions From Tyk 4.0, you can use PostgreSQL as your datastore. We support the following versions: You can also use the following as a drop in replacement for PostgreSQL: For POC, you can also use the following as replacement:
  • SQLite 3.x

​
Migrating from an existing MongoDB instance

For v4.0 we have provided a migration command that will help you migrate all data from the main storage layer (APIs, Policies, Users, UserGroups, Webhooks, Certificates, Portal Settings, Portal Catalogs, Portal Pages, Portal CSS, etc.).
The migration tool will not migrate any Logs, Analytics, or Uptime analytics data.
  1. Make sure your new SQL platform and the existing MongoDB instance are both running
  2. Configure the main part of the storage section of your tyk-analytics.conf:
{
...
  "storage": {
    ...
    "main": {
      "type": "postgres",
      "connection_string": "user=root password=admin database=tyk-demo-db host=tyk-db port=5432"
    }
  }
}
  1. Run the following command:
./tyk-analytics migrate-sql
You will see an output listing the transfer of each database table. For example: Migrating 'tyk_apis' collection. Records found: 7.
  1. You can now remove your mongo_url (or TYK_DB_MONGOURL environment variable) from your tyk-analytics.conf
  2. Restart your Tyk Dashboard

​
PostgreSQL Sizing Guidelines

The aggregate record size depends on the number of APIs and Keys you have. Each counter size is ~50b, and every aggregated value has its own counter. So an hourly aggregate record is computed like this: 50 * active_apis + 50 * api_versions + 50 * active_api_keys + 50 * oauth_keys, etc. The average aggregate record size (created hourly) on our cloud is about ~ 40KB (a single record includes all the aggregate stats mentioned above). So for 1 million requests per day, it will generate 1KB * 1M request stats (1GB) + 24 * 40KB aggregate stats (~1MB). Per month: 30GB request logs + 30MB aggregate logs PostgreSQL Database Storage Calculator You can calculate your PostgreSQL storage requirements by entering your known values in the middle section of the calculator settings below:

​
Database Storage Calculator

Use this calculator to estimate the storage requirements for your database when using Tyk. This tool helps you plan your database infrastructure by calculating storage needs based on your API traffic, analytics retention, and configuration data. The calculator considers factors like requests per second, analytics time-to-live, and the number of APIs and policies you manage. Results show the total storage requirement accounting for your specified utilization threshold.