Seeking Clarification on MSP API Retention Limits and Bulk Data Export Workarounds.

Hi everyone,

Over the past 30 days, my Firewalla Gold has logged approximately 3.5 million network flows. For deeper security auditing, pattern analysis, and local archiving, I am looking to ingest this entire historical dataset into my on-premise data server.

However, after extensive testing and developing a custom integration against the Firewalla MSP environment, I have encountered a few structural constraints that seem to severely limit automated data ownership. I would like to verify if my findings are correct, or if I am misinterpreting the platform's architecture:

1. The Cloud API Retention Window: While the MSP web portal beautifully retains 30 to 90 days of history, my programmatic requests to the /v2/flows endpoint appear to hit a hard filtering ceiling. Any queries utilizing timestamps older than 24 hours return an empty dataset (0 results). Is the API gateway intentionally restricted to a rolling 24-hour cache, regardless of the MSP tier retention settings?

2. Inconsistent API Synchronization: During high-frequency polling, the API responses exhibit symptoms of eventual consistency across the cloud backend (distributed server clusters). We frequently observe fluctuating pagination behaviors, where successive requests alternatively yield chunks of data or empty results before catching up. This makes a reliable data pipeline highly unpredictable.

3. Manual CSV Export Constraints: The manual export utility via the MSP UI is capped at 50,000 rows per file, combined with a strict rate limit of 20 downloads per 48 hours. For a volume of 3.5 million rows, extracting this data manually would turn into a tedious, week-long administrative task.

I might entirely be missing a vital piece of documentation or an alternative workflow.

Are my assumptions regarding these API and export limits accurate? Are there any official, streamlined endpoints or asynchronous bulk-export methods (e.g., a /v2/exports request) designed for full data ingestion that I have overlooked?

Any insights, workarounds, or official clarifications would be greatly appreciated!

Best regards,
Ilco Louis