In order for the Databricks cluster to be able to communicate with Hopsworks, clients running on Databricks need to be able to access a Hopsworks API key.
For instructions on how to generate an API key follow this user guide. For the Databricks integration to work make sure you add the following scopes to your API key:
- featurestore
- project
- job
- kafka
!!! hint "API key as Argument" To get started quickly, without saving the Hopsworks API in a secret storage, you can simply supply it as an argument when instantiating a connection:
import hopsworks
project = hopsworks.login(
host='my_instance', # DNS of your Feature Store instance
port=443, # Port to reach your Hopsworks instance, defaults to 443
project='my_project', # Name of your Hopsworks Feature Store project
api_key_value='apikey', # The API key to authenticate with Hopsworks
)
fs = project.get_feature_store() # Get the project's default feature store
Continue with the configuration guide to finalize the configuration of the Databricks Cluster to communicate with the Hopsworks Feature Store.