Hive Security, in depth: authentication, wire protection, the metastore side door, secrets, hardening and audit

The threat model

Start by writing down what you are defending against, because each threat needs a different control. Impersonation: a client claims to be someone else, which matters when authentication is off or weak. Eavesdropping and tampering: queries, results and passwords cross the network, and on a flat network anyone can capture them. Bypass: a user who is denied a table in HiveServer2 reads it through the metastore and the file system instead. Privilege escalation: an authenticated user runs commands or functions that act with the server's identity, such as reading local files on the HiveServer2 host. Credential theft: passwords and keytabs sitting in world-readable configuration. Repudiation: nobody can say afterwards who read what.

The diagram's lower-left arrow is the one teams forget: Spark and scripts that talk to the metastore directly and then read files themselves. Everything in the rest of this article is about making each arrow authenticated, encrypted, authorized and logged.

Authentication at HiveServer2

The hive.server2.authentication property selects how HiveServer2 establishes identity. It is the foundation for every other control, because authorization and audit are only as good as the user name they are given.

Kerberos is the natural choice inside a Hadoop estate, because HDFS, YARN and the metastore already speak it; see the Hadoop Kerberos article for how tickets, principals and keytabs work. The _HOST token in the principal is replaced with the server's canonical host name at startup, so one configuration serves every HiveServer2 instance.

<!-- hive-site.xml on HiveServer2: Kerberos over binary transport, encrypted with SASL -->
<property><name>hive.server2.authentication</name><value>KERBEROS</value></property>
<property><name>hive.server2.authentication.kerberos.principal</name>
          <value>hive/_HOST@EXAMPLE.COM</value></property>
<property><name>hive.server2.authentication.kerberos.keytab</name>
          <value>/etc/security/keytabs/hive.service.keytab</value></property>
<property><name>hive.server2.thrift.sasl.qop</name><value>auth-conf</value></property>
<property><name>hive.server2.enable.doAs</name><value>false</value></property>

LDAP is simpler for users of desktop tools, but it sends a password, so it must only run over TLS. Prefer ldaps:// for the HiveServer2-to-directory connection as well, and restrict which users and groups may log in with the LDAP filter properties rather than accepting everyone in the directory.

<!-- hive-site.xml: LDAP passwords over http transport, protected by TLS -->
<property><name>hive.server2.authentication</name><value>LDAP</value></property>
<property><name>hive.server2.authentication.ldap.url</name>
          <value>ldaps://ldap.example.com:636</value></property>
<property><name>hive.server2.authentication.ldap.baseDN</name>
          <value>ou=people,dc=example,dc=com</value></property>
<property><name>hive.server2.transport.mode</name><value>http</value></property>
<property><name>hive.server2.thrift.http.path</name><value>cliservice</value></property>
<property><name>hive.server2.use.SSL</name><value>true</value></property>
<property><name>hive.server2.keystore.path</name><value>/etc/hive/tls/hs2.jks</value></property>

One property deserves a warning of its own: hive.server2.trusted.domain. Its description in HiveConf says authentication is skipped for any connection whose host name ends with the configured value. It exists for trusted proxies, and set carelessly it switches authentication off for a whole domain. Audit it as a finding whenever it is non-empty.

Transports and wire protection

HiveServer2 speaks Thrift over one of two transports, chosen by hive.server2.transport.mode: binary (a raw socket, by default on port 10000), http (Thrift inside HTTP, which passes through load balancers and gateways and by default uses port 10001), or all for both. Each has its own way to be encrypted.

On the binary transport with Kerberos, the SASL layer can protect the stream. hive.server2.thrift.sasl.qop takes auth (authentication only, the default), auth-int (adds integrity checks) or auth-conf (adds encryption). The default authenticates the user and then sends every query and result in clear text. Set auth-conf, and clients must request the same level with saslQop in the JDBC URL, or the connection fails with a SASL negotiation error.

The alternative is TLS, enabled with hive.server2.use.SSL and a keystore. TLS is the only option for LDAP, and on http transport it is the natural choice; Kerberos then runs as SPNEGO inside HTTP. Pick one encryption mechanism per listener and make it mandatory, rather than offering encrypted and plain endpoints side by side, because clients drift to whichever works first.

# Kerberos: get a ticket first, then name the SERVICE principal in the URL
kinit alice@EXAMPLE.COM
beeline -u "jdbc:hive2://hs2.example.com:10000/default;principal=hive/_HOST@EXAMPLE.COM;saslQop=auth-conf"

# LDAP over http + TLS: the password travels inside TLS
beeline -u "jdbc:hive2://hs2.example.com:10001/default;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/hive/tls/truststore.jks" -n alice -p

# Negative tests you should script and run after every change
beeline -u "jdbc:hive2://hs2.example.com:10000/default"      # must fail: no principal
kdestroy && beeline -u "jdbc:hive2://hs2.example.com:10000/default;principal=hive/_HOST@EXAMPLE.COM"  # must fail

The negative tests at the end matter as much as the positive ones. A hardening change that accidentally leaves authentication off looks identical to a working one when you only test with valid credentials.

The metastore: the side door

The metastore is a Thrift service that can create and drop tables, change their locations and alter partitions. If it accepts unauthenticated calls, anyone on the network can repoint a table at a directory they control or drop it outright, and no SQL policy in HiveServer2 is consulted. Turn on SASL so every caller must present a Kerberos identity.

<!-- hive-site.xml on the Metastore: authenticate every Thrift caller -->
<property><name>hive.metastore.sasl.enabled</name><value>true</value></property>
<property><name>hive.metastore.kerberos.principal</name><value>hive/_HOST@EXAMPLE.COM</value></property>
<property><name>hive.metastore.kerberos.keytab.file</name>
          <value>/etc/security/keytabs/hive.service.keytab</value></property>

Authentication alone does not decide what a caller may do. Storage-based authorization on the metastore checks each metadata change against the permissions of the table's directory, which is the control for clients that bypass HiveServer2; its configuration is shown in the authorization article. Firewall the metastore port so only HiveServer2, the compute clusters and administrators can reach it at all; defence in depth is cheap here because the list of legitimate callers is short. Jobs that run on a cluster without the user's Kerberos ticket use delegation tokens issued by the metastore, so token lifetimes and renewers belong in the same review. The metastore article covers its internals.

The metastore's own database holds every table definition. Its connection password traditionally sits in hive-site.xml as javax.jdo.option.ConnectionPassword, readable by anyone who can read the configuration directory, including every job that ships configuration with it.

Secrets out of configuration

# Move the metastore database password out of hive-site.xml into a keystore
hadoop credential create javax.jdo.option.ConnectionPassword \
    -provider jceks://file/etc/hive/conf/hive.jceks
chmod 400 /etc/hive/conf/hive.jceks && chown hive:hive /etc/hive/conf/hive.jceks

<!-- then point Hive at the provider and delete the plaintext property -->
<property><name>hadoop.security.credential.provider.path</name>
          <value>jceks://file/etc/hive/conf/hive.jceks</value></property>

The Hadoop credential provider API stores secrets in a keystore file and lets Hive look them up by the same property name, so the plaintext value can be removed. Protect the keystore with file permissions; by default a JCEKS store is protected only by a well-known password, so its security is the file's permissions, not the format. HiveConf also keeps two lists that limit leakage from inside a session: hive.conf.hidden.list names properties, including the metastore password and keystore passwords, whose values are hidden from users, and hive.conf.restricted.list names properties a session may not change with SET, which includes the authorization and authenticator managers and the LDAP settings. Add any site-specific secret or security switch to both lists rather than trusting their defaults to cover it.

Authorization in one paragraph

Once identity is trustworthy, decide what each user may do. Hive offers SQL-standard authorization inside HiveServer2, storage-based authorization on the metastore, and Apache Ranger for central policies with column masking and row filtering; see the Ranger article for the policy engine itself. The decision that shapes everything is hive.server2.enable.doAs. With doAs on, queries run as the end user and HDFS permissions do the enforcing; with it off, queries run as the hive service user, files can be locked away from ordinary users, and HiveServer2 policies become the single gate. Most governed deployments turn doAs off and use Ranger. The trade-offs are covered in full in the authorization article.

Data at rest

Authentication and authorization protect data from users; encryption at rest protects it from stolen disks, mis-scoped backups and storage administrators. On HDFS, transparent encryption places table directories in encryption zones whose keys live in the Hadoop KMS, as described in the encryption zones article. Hive is zone-aware. Files cannot be renamed across zone boundaries, so when HDFS encryption is enabled the query compiler stages intermediate files inside the encryption zone of the most strongly encrypted table the query touches. If that table is read-only it falls back to the scratch directory, and refuses the query when the scratch directory is unencrypted or more weakly encrypted than the table. The exposure that remains is local disk: the HiveServer2 host's local scratch directory (hive.exec.local.scratchdir) and downloaded resources (hive.downloaded.resources.dir) sit outside HDFS, so put them on encrypted volumes. On object stores, use the provider's server-side encryption with customer-managed keys and restrict who may use the key, because anyone who can use the key can read the data.

Hardening the surface beyond SQL

<!-- Narrow the non-SQL command surface (default: set,reset,dfs,add,list,delete,reload,compile,llap) -->
<property><name>hive.security.command.whitelist</name><value>set,reset</value></property>
<!-- Built-in UDFs that call arbitrary Java methods or read local files -->
<property><name>hive.server2.builtin.udf.blacklist</name>
          <value>reflect,reflect2,java_method,in_file</value></property>
<!-- Protect the web UI the same way as the JDBC endpoint -->
<property><name>hive.server2.webui.use.ssl</name><value>true</value></property>
<property><name>hive.server2.webui.use.spnego</name><value>true</value></property>

An authenticated user can do more than run queries. The dfs command runs file system operations, add pulls jars and files into the session, and compile compiles source into a function on the server. With doAs off these act with the hive service identity, which can read every managed table. Cut the command whitelist down to what your users need. The functions reflect, reflect2 and java_method call arbitrary Java methods, and in_file reads files local to the server; blocklist them unless a documented use needs them. Protect the HiveServer2 web UI, which shows queries and configuration, with TLS and SPNEGO just like the JDBC endpoint, and run HiveServer2 and the metastore under dedicated service accounts with no interactive login.

Audit

Answering who read what needs events from more than one place. Ranger's Hive plugin records each authorization decision with the user, resource, action and result. HDFS audit logs show file access, and with doAs off every access appears as the hive user, so the Hive-level audit is the only record of the real person. HiveServer2 logs sessions and queries. Ship all of them to a central store, keep them outside the cluster's own administrators' reach, and alert on patterns that indicate probing: repeated denials for one user, access outside working hours, or metastore calls from hosts that are not on the allowed list.

Failure modes

• Clock skew. Kerberos rejects tickets when clocks differ by more than the allowed skew; run time synchronization on every node and client.
• Principal and host name mismatch. Behind a load balancer the client asks for a ticket for the balancer's name; either give the service a principal for that name or use ZooKeeper service discovery so clients connect to real hosts.
• QoP mismatch. Server at auth-conf, client at the default: the connection fails during SASL negotiation, not at login, which misleads debugging.
• Expired tickets in long sessions. BI servers need keytabs and automatic renewal, not a ticket from a person's login.
• Unsecured metastore. The most common real gap: HiveServer2 is locked down, the metastore port is open and unauthenticated.
• Plaintext secrets in shipped configuration. Job configurations and support bundles copy hive-site.xml; anything in it should be assumed disclosed.

What to do next

1. Draw your own version of the attack-surface diagram and list every client that reaches the metastore and the storage directly.
2. Set HiveServer2 authentication to Kerberos or LDAP, make encryption mandatory with auth-conf or TLS, and script the negative connection tests.
3. Enable SASL on the metastore, firewall its port to known callers, and apply storage-based authorization.
4. Move every password in hive-site.xml into a credential provider and extend the hidden and restricted lists.
5. Trim the command whitelist, blocklist the reflection and file UDFs, and confirm hive.server2.trusted.domain is empty.
6. Centralize Ranger, HiveServer2 and HDFS audit logs and add alerts for repeated denials and unknown metastore callers.