In today’s world, given the pace at which data operates, we need a tool that can help us to generate reports faster and bring out insights within milliseconds. In order to solve this challenge, several companies have started utilising a few Business Intelligence (BI) tools such as Tableau/Power BI/Superset/Looker/Qlikview, etc. We at Blinkit have also moved away from the traditional way of reporting via spreadsheets to a more scalable and robust tool — Tableau.

Earlier, we had no single source of truth for metrics; it took a significant amount of manual effort to compile data in spreadsheets, and we were frequently limited by our local memory.

Theoretically, Tableau can handle data up to ∞ rows and columns. However, it slowly begins to slow down as it renders calculations, fields, and formulas. At Blinkit, tracking supply and consumer metrics in real-time is the need of the hour. The faster we track, the faster we decide and move forward.

Given below are some hacks that enabled us to achieve a significant improvement in the performance of our dashboards. The effects have been noted across various aspects of our operations, including but not limited to:

• Reduction in the average render/load time by over 50% in the last six months.
• Over a three-fold improvement in throughput in traffic to views in the last three months.
• A 50% reduction in the extract time over the last six months.

Check performance recording
This is the first step in identifying the problem. The performance recording gives information about key events as you interact with a workbook. Further details can be found here.

Generating extract
To speed up extract generation, consider importing only some data from the original data source by using filters or importing a limited sample of data.

Connecting to data sources
Slow connections could be due to network issues or issues with the database server.

Executing Queries

• For live connections, if queries are taking too long, it could be because the underlying data structure isn’t optimised for Tableau. Consult your Database server’s documentation. As an alternative, also consider using extracts as opposed to live connections.
• For extracts, if queries are taking too long, review your filters. If you have a lot of filters, check if a context filter would make more sense. If you have a dashboard that uses filters, consider using action filters, which can help with performance.

Always keep your marks lower
It is not advisable to keep large columns*rows (or marks) in the same view as it would lead to a rendering effect while loading the view.

1. Create hierarchies on dimensions and keep a summary view at default. Use +/- sign when need to expand/collapse, respectively.
2. Don’t force everything into a single view; instead, break them into multiple views and use filters.
3. Keep limited dimensions on details in the marks shelf.
4. Display data using charts/graphs — avoid tabular view. Aggregated charts are far easier to consume and help convey a story.

Push complicated calculations to backend ETLs
We at Blinkit, maintain a layered architecture in our data warehouse. We have scheduled ETLs that power Tableau ingestion tables on an hourly or daily basis. We try to do most of the complex calculations within these ETLs and perform only non-additive/dynamic calculations on Tableau.

1. Wherever possible, aggregate all your rows in the backend itself. Tableau loves feeding aggregated calculations (as shown in Fig. 4). If required, use MIN or MAX over AVG.
2. Avoid using string calculations wherever possible. Datatypes in ascending order of computation time–Boolean<Int<Float<Date<DateTime<String.
3. Avoid using table calculations and LOD (level of details) functions (Fig. 5).

Use data extracts over live connection
Tableau allows two different connections, Live and Extract. While building views, always keep your data source connection as an extract; these are cached. Live connections are immense slow when working with huge data sets.

Incremental refresh over full refresh
Incremental refresh appends the record in a table based on the key chosen (such as at_date_ist in Fig. 8). Whenever the previous records don’t get updated, it's advisable to use an incremental refresh, which will decrease the refresh time of your workbook.

Hide all unused fields
Hide unused columns (measures/dimensions) in order to minimise extract refresh time or custom SQL query time.

Conclusion
By following the best practices discussed above in this blog, you can create a fast and efficient dashboard that provides your users with valuable insights.

Akash is a Senior BI Engineer at Blinkit. Connect with him on LinkedIn.

Thanks for reading, and happy dashboarding!