Open Distro for Elasticsearch(Windows)

Ashok Selva

11 Mar 2022

Open Distro for Elasticsearch provides a powerful, easy-to-use event monitoring, security, and alerting system, enabling you to monitor your data and send notifications automatically to your stakeholders. With an intuitive Kibana interface and powerful API, it is easy to set up and manage alerts.
Available Plugins in Open Distro for Elasticsearch :

Security
Job Scheduler
Alerting
SQL
Reports Scheduler
Index State Management
KNN
Anomaly Detection

Performance Analyzer

Elasticsearch version	Plugin versions
7.10.0	opendistro-anomaly-detection 1.12.0.0 opendistro-job-scheduler 1.12.0.0 opendistro-knn 1.12.0.0 opendistro_alerting 1.12.0.2 opendistro_index_management 1.12.0.1 opendistro_performance_analyzer 1.12.0.0 opendistro_security 1.12.0.0 opendistro_sql 1.12.0.0 opendistro-reports-scheduler 1.12.0.0
7.9.1	opendistro-anomaly-detection 1.10.1.0, 1.11.0.0 opendistro-job-scheduler 1.10.1.0, 1.11.0.0 opendistro-knn 1.10.1.0, 1.11.0.0 opendistro_alerting 1.10.1.2, 1.11.0.1 opendistro_index_management 1.10.1.1, 1.11.0.0 opendistro_performance_analyzer 1.10.1.0, 1.11.0.0 opendistro_security 1.10.1.0, 1.11.0.0 opendistro_sql 1.10.1.1, 1.11.0.0
7.8.0	opendistro-anomaly-detection 1.9.0.0 opendistro-job-scheduler 1.9.0.0 opendistro-knn 1.9.0.0 opendistro_alerting 1.9.0.0 opendistro_index_management 1.9.0.0 opendistro_performance_analyzer 1.9.0.1 opendistro_security 1.9.0.0 opendistro_sql 1.9.0.0
7.7.0	opendistro-anomaly-detection 1.8.0.0 opendistro-job-scheduler 1.8.0.0 opendistro-knn 1.8.0.0 opendistro_alerting 1.8.0.0 opendistro_index_management 1.8.0.0 opendistro_performance_analyzer 1.8.0.0 opendistro_security 1.8.0.0 opendistro_sql 1.8.0.0
7.6.1	opendistro-anomaly-detection 1.7.0.0 opendistro-job-scheduler 1.7.0.0 opendistro-knn 1.7.0.0 opendistro_alerting 1.7.0.0 opendistro_index_management 1.7.0.0 opendistro_performance_analyzer 1.7.0.0 opendistro_security 1.7.0.0 opendistro_sql 1.7.0.0
7.4.2	opendistro-job-scheduler 1.4.0.0 opendistro-knn 1.4.0.0 opendistro_alerting 1.4.0.0 opendistro_index_management 1.4.0.0 opendistro_performance_analyzer 1.4.0.0 opendistro_security 1.4.0.0 opendistro_sql 1.4.0.0
7.3.2	opendistro-job-scheduler 1.3.0.0 opendistro_alerting 1.3.0.1 opendistro_index_management 1.3.0.1 opendistro_performance_analyzer 1.3.0.0 opendistro_security 1.3.0.0 opendistro_sql 1.3.0.0
7.2.1	opendistro-job-scheduler 1.2.1 opendistro_alerting 1.2.1.0 opendistro_performance_analyzer 1.2.1.0 opendistro_security 1.2.1.0 opendistro_sql 1.2.1.0
7.2.0	opendistro-job-scheduler 1.2.0 opendistro_alerting 1.2.0.0 opendistro_performance_analyzer 1.2.0.0 opendistro_security 1.2.0.0 opendistro_sql 1.2.0.0
7.1.1	opendistro-job-scheduler 1.1.0 opendistro_alerting 1.1.0.0 opendistro_performance_analyzer 1.1.0.0 opendistro_security 1.1.0.0 opendistro_sql 1.1.0.0
7.0.1	opendistro-job-scheduler 1.0.0 opendistro_alerting 1.0.0.0 opendistro_performance_analyzer 1.0.0.0 opendistro_security 1.0.0.2 opendistro_sql 1.0.0.0
6.8.1	opendistro_alerting 0.10.0.0 opendistro_performance_analyzer 0.10.0.0 opendistro_security 0.10.0.0 opendistro_sql 0.10.0.0
6.7.1	opendistro_alerting 0.9.0.0 opendistro_performance_analyzer 0.9.0.0 opendistro_security 0.9.0.0 opendistro_sql 0.9.0.0
6.6.2	opendistro_alerting 0.8.0.0 opendistro_performance_analyzer 0.8.0.0 opendistro_security 0.8.0.0 opendistro_sql 0.8.0.0
6.5.4	opendistro_alerting 0.7.0.0 opendistro_performance_analyzer 0.7.0.0 opendistro_security 0.7.0.1 opendistro_sql 0.7.0.0

Note: In this article, we'll only focus on the security Plugin for the Windows Platform.

Prerequisites :

Elasticsearch(To install plugins manually, you must have the exact OSS version of Elasticsearch installed (for example, 6.6.2 and not 6.6.1).

Step 1:
Navigate to the Elasticsearch directory (most likely, it is elasticsearch/bin), and run the install command for each plugin.

elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-security/opendistro_security-1.12.0.0.zip

Step 2:

Install certificates, to do this navigate to (most likely, it is elasticsearch/plugins/opendistrosecurity/tools), and run the shell script(install_demo_configuration.sh)

install_demo_configuration.sh

Step 3:
Since we are trying to integrate an open-source plugin that is not developed by ELK, we need to turn off the native x-pack security. To do so navigate to the config directory and open elasticsearch.yml (most likely, it is elasticsearch/config/elasticsearch.yml).Then add the following line in that file.

xpack.security.enabled: false

Step 4:
Start Elasticsearch and try to hit https://localhost:9200 , you will get a response like this

It means , open distro security plugin is installed and is not initialized to do authentication and authorization. To initialize it you have run the securityadmin.bat with required arguments

NOTE: The path provided here for cert files will work if you haven't moved your cert files to some other directory, you can always refer to the directory where your cert files are stored.

securityadmin.bat  -cd ../securityconfig/ -nhnv -cacert ../../../config/root-ca.pem  -cert ../../../config/kirk.pem   -key ../../../config/kirk-key.pem   -h localhost

The followings are the arguments that you might need for advanced configurations.

usage: securityadmin.sh [-arc] [-b
ackup <folder>] [-cacert <file>] [-cd
       <directory>] [-cert <file>] [-cn <clustername>] [-dci] [-dg] [-dra]
       [-ec <cipers>] [-ep <protocols>] [-er <number of replicas>] [-era]
       [-esa] [-f <file>] [-ff] [-h <host>] [-i <indexname>] [-icl] [-key
       <file>] [-keypass <password>] [-ks <file>] [-ksalias <alias>]
       [-kspass <password>] [-kst <type>] [-migrate <folder>] [-mo
       <folder>] [-nhnv] [-noopenssl] [-nrhn] [-p <port>] [-prompt] [-r]
       [-rev] [-rl] [-si] [-sniff] [-t <file-type>] [-ts <file>] [-tsalias
       <alias>] [-tspass <password>] [-tst <type>] [-us <number of
       replicas>] [-vc <version>] [-w]
 -arc,--accept-red-cluster                      Also operate on a red
                                                cluster. If not specified
                                                the cluster state has to
                                                be at least yellow.
 -backup <folder>                               Backup configuration to
                                                folder
 -cacert <file>                                 Path to trusted cacert
                                                (PEM format)
 -cd,--configdir <directory>                    Directory for config files
 -cert <file>                                   Path to admin certificate
                                                in PEM format
 -cn,--clustername <clustername>                Clustername (do not use
                                                together with -icl)
 -dci,--delete-config-index                     Delete
                                                '.opendistro_security'
                                                config index and exit.
 -dg,--diagnose                                 Log diagnostic trace into
                                                a file
 -dra,--disable-replica-autoexpand              Disable replica auto
                                                expand and exit
 -ec,--enabled-ciphers <cipers>                 Comma separated list of
                                                enabled TLS ciphers
 -ep,--enabled-protocols <protocols>            Comma separated list of
                                                enabled TLS protocols
 -er,--explicit-replicas <number of replicas>   Set explicit number of
                                                replicas or autoexpand
                                                expression for
                                                .opendistro_security index
 -era,--enable-replica-autoexpand               Enable replica auto expand
                                                and exit
 -esa,--enable-shard-allocation                 Enable all shard
                                                allocation and exit.
 -f,--file <file>                               file
 -ff,--fail-fast                                fail-fast if something
                                                goes wrong
 -h,--hostname <host>                           Elasticsearch host
                                                (default: localhost)
 -i,--index <indexname>                         The index Open Distro
                                                Security uses to store the
                                                configuration
 -icl,--ignore-clustername                      Ignore clustername (do not
                                                use together with -cn)
 -key <file>                                    Path to the key of admin
                                                certificate
 -keypass <password>                            Password of the key of
                                                admin certificate
                                                (optional)
 -ks,--keystore <file>                          Path to keystore
                                                (JKS/PKCS12 format
 -ksalias,--keystore-alias <alias>              Keystore alias
 -kspass,--keystore-password <password>         Keystore password
 -kst,--keystore-type <type>                    JKS or PKCS12, if not
                                                given we use the file
                                                extension to dectect the
                                                type
 -migrate <folder>                              Migrate and use folder to
                                                store migrated files
 -mo,--migrate-offline <folder>                 Migrate and use folder to
                                                store migrated files
 -nhnv,--disable-host-name-verification         Disable hostname
                                                verification
 -noopenssl,--no-openssl                        Do not use OpenSSL even if
                                                available (default: use it
                                                if available)
 -nrhn,--disable-resolve-hostname               Disable DNS lookup of
                                                hostnames
 -p,--port <port>                               Elasticsearch transport
                                                port (default: 9300)
 -prompt,--prompt-for-password                  Prompt for password if not
                                                supplied
 -r,--retrieve                                  retrieve current config
 -rev,--resolve-env-vars                        Resolve/Substitute env
                                                vars in config with their
                                                value before uploading
 -rl,--reload                                   Reload the configuration
                                                on all nodes, flush all
                                                Open Distro Security
                                                caches and exit
 -si,--show-info                                Show system and license
                                                info
 -sniff,--enable-sniffing                       Enable
                                                client.transport.sniff
 -t,--type <file-type>                          file-type
 -ts,--truststore <file>                        Path to truststore
                                                (JKS/PKCS12 format)
 -tsalias,--truststore-alias <alias>            Truststore alias
 -tspass,--truststore-password <password>       Truststore password
 -tst,--truststore-type <type>                  JKS or PKCS12, if not
                                                given we use the file
                                                extension to dectect the
                                                type
 -us,--update_settings <number of replicas>     Update the number of Open
                                                Distro Security index
                                                replicas, reload
                                                configuration on all nodes
                                                and exit
 -vc,--validate-configs <version>               Validate config for
                                                version 6 or 7 (default 7)
 -w,--whoami                                    Show information about the
                                                used admin certificate

Step 5:
Now try to access https://localhost:9200, if you get a prompt asking for username and password. Hurrrrayyy🎊🎉!! we have implemented the basic auth using open distro.

The default username will be admin and password is admin.

What if you don't need basic authentication….Surprise 🎊🎉 we can implement token based authentication using Open Distro.

To implement token based authentication follow this steps:

Navigate to elasticsearch/plugins/opendistrosecurity/securityconfig and open config.yml make the changes as below.

jwt_auth_domain:
      description: "Authenticate via Json Web Token"
      http_enabled: true
      transport_enabled: true
      order: 0
      http_authenticator:
        type: jwt
        challenge: false
        config:
          signing_key: "IXIENkVkTX6+QS1NVntGWIvYa7h8JC5ONZpegpkuUw0="
          jwt_header: "Authorization"
          jwt_url_parameter: null
          roles_key: "roles"
          subject_key: "sub"
      authentication_backend:
        type: noop

Note: If you only need token based authentication and not basic authentication , then make http_enabled and transport_enabled to false in basic_internal_auth_domain section.

Once you are done with the changes save the file and run the securityadmin.bat (specified in step 4).
If you got this message “Done with success” .Then we can test our implementation.
Open https://jwt.io/. Then create a token with following payload.

{
  "sub": "admin",
  "roles": "admin",
  "exp": 1616239022
}

change the exp value in payload accordingly.

Make sure you have added the signing key as well like in the above picture
copy the token and try hitting elasticsearch in postman with Bearer token (copied jwt token).

Likewise, we can also enable proxy-based authentication.
Here is the reference link for detailed information about each plugin Open Distro.