Microsoft Azure provides a high degree of flexibility around the security of storage accounts by enabling various configurations that can be applied to secure data both at rest and during transit. These security building blocks put together to enable a highly secure data infrastructure that can be constructed based on organisational security and governance requirements. Best practices for securing Azure Storage Accounts can be grouped into the following:
- 1. Identity and Access Control considerations
- 2. Data Protection considerations
- 3. Networking considerations
Identity and Access Control
The basic level of security that can be applied to authorize access to data in Azure Active Directory enabled access management. Examples of other security elements that can be super-imposed on Azure AD to secure access to data include enabling Shared Access Signature (SAS) keys to delegate fine-grained and more restrictive access (typically for external entities requiring access to data over a specific time frame). Periodic regeneration of access keys, restricting access to individuals and devices over a specific network configuration and enforcing the principle of least privilege while provisioning of access is examples of other best practices that bolster data security from an Identity and Access Control perspective.
Data Protection, Redundancy and Disaster Recovery
Another important Lakehouse Security, Data Transit, and Disaster Recovery step is to consider the data protection element of data security. This includes enabling Role-based Access Control (RBAC) to containers and blobs, enabling data redundancy options (LRS, ZRS, GRS, GZRS) for disaster recovery, data backup considerations, and enabling soft-delete on blobs for recovery, and TLS considerations. This also includes securing data over the transfer by ensuring that all traffic to the storage account is routed over an HTTPS protocol.
Networking and Data Transfer
Some of the best practices to be considered from a Networking aspect include encompassing Azure services within Virtual Networks and subnets and ensuring the traffic to the Storage Account is restricted to a specific range of IP addresses. Setting up Network Security groups and firewall rules for traffic permitted for ingress into and egress from the Storage Account are some of the other networking best practices to be considered for securing data in Storage Accounts.
In addition to the aforementioned, there are various different options that Microsoft Azure provides to configure Data Storage Infrastructure security. Each Azure service has inherent security features that can be coupled together to secure highly sensitive data. Some examples include setting up Dynamic data masking to restrict access to sensitive data, locking down of sensitive data in the Bronze layer of the Data Lakehouse, and de-identification/masking/encryption of data before ingress into the Lakehouse.
Get in touch with AI Consulting Group to understand more about best practices on Lakehouse Security, Data Transit, and Disaster Recovery based on your organisational needs.