Data access options: BI Staging, API, and Front-end exports

Updated 2026-02-03 08:35:11 UTC Created 2025-10-07 00:05:00 UTC

Summary: This guide explains when to use the BI staging server, the API, and front-end exports. For large organisations and large date ranges, front-end exports are not the right solution.

Contents

Overview

Easy Employer provides a Business Intelligence (BI) staging server as an optional upgrade. This SQL Server database allows customers to connect directly and extract reporting-ready data for dashboards and analytics. Easy Employer does not directly integrate with third-party BI reporting platforms; instead, the BI staging server provides structured, reporting-ready data that your BI tools can consume.

TIP: Use front-end exports for small, ad-hoc pulls. Use BI staging for big extracts and analytics at scale. Use the API for transactional or workflow use cases.

BI staging server and BI reporting solution

  • BI staging server: A curated database that consolidates and prepares data for reporting and analysis. It provides structured, stable schemas that can be queried and extracted by your BI tools.
  • BI reporting solution (for example Power BI): A visualisation and analytics tool that connects to data sources (including the BI staging server) to create dashboards and reports.

What data is available

The BI staging database includes multiple datasets from the Easy Employer application. See the data dictionary: Easy Employer – BI staging server – data dictionary

Data refresh frequency

Data syncs occur every three hours. We will assess your organisation’s database to determine the optimal number of daily syncs and timings.

BI staging vs API

What each is for

Option Purpose
BI staging server Read-only, reporting-ready database designed for dashboards, ad-hoc analysis, and big extracts.
API connection Transactional interface to the live app for creating, reading, and updating specific records in near real time.

When to use which

Use this When you need Notes
BI staging server
  • Big extracts and self-service analytics
  • Stable, read-only access for BI tools with minimal engineering
  • Optimised for large reads and heavy queries (millions of rows) without impacting the production app
  • Stable schemas tuned for reporting; read-only reduces risk to production
API connection
  • Real-time reactions in a workflow (for example create or update entities)
  • Integrations with other apps, automations, or on-demand fetches
  • Custom data shapes or fields not exposed in the staging layer
  • Rate limits and pagination can make large pulls slow and add load
  • Schemas can evolve with features; you own transform and validation logic

Front-end exports: where they fit and limits

Front-end exports run in your browser session. They are convenient for small to medium datasets and quick CSVs. They are not designed for organisation-wide extracts, multi-month ranges, or very wide tables.

NOTE: Front-end exports are session-bound and sensitive to browser memory, tab sleep, and connection drops. For large extracts, use the BI staging server.

Why front-end exports struggle on large data

  • Browser limits: Memory constraints, tab suspensions, and time-outs.
  • Network volatility: Long-running downloads are disrupted by brief connection issues.
  • Client-side work: Formatting and aggregations performed in the UI add processing time.
  • User actions: Closing the tab or putting the laptop to sleep interrupts the export.

Scale thresholds and practical limits

  • Up to roughly 50k rows: Front-end export is usually fine.
  • Around 50k–100k rows: Consider BI staging; front-end may succeed but can be slow.
  • 100k+ rows or wide tables: Use BI staging. Do not rely on front-end exports.
  • Multi-month, multi-site ranges: Use BI staging for reliability and performance.
TIP: Prefer CSV for ad-hoc exports. For recurring analysis, use BI staging and connect your BI tool.

Troubleshooting and tips

  • Narrow your filters: Restrict date ranges or scope to a site, team, or user group.
  • Export off-peak: Reduce contention and improve responsiveness.
  • Avoid leaving long exports unattended: Keep the tab active to prevent sleep or suspension.
  • Use BI for scheduled analysis: Replace repeated manual exports with a BI dataset.

FAQ

Can we increase the front-end time-out?
Higher time-outs do not address browser memory and stability constraints. For large extracts, use BI staging.

Why not use the API for big pulls?
APIs are paginated and rate-limited. They are ideal for transactional workflows, not bulk analytics. BI staging is tuned for heavy reads.

What if we need custom fields?
Check the BI schema first. If a field is not modelled, consider a lightweight API step to enrich BI outputs downstream, or contact us to see if it can be added.

BI staging server cost

If you are interested in this solution, contact support@easyemployer.com and we will provide monthly pricing tailored to your organisation.