In this section, we will start interacting directly with Elasticsearch by using the Index API. This is an extremely important step because it allows us to create, inspect, and delete indices, which form the foundation of everything we do in Elasticsearch.
Opening Kibana Dev Tools
To begin, we need access to the Kibana Dev Tools console, which is the primary interface used for interacting with Elasticsearch through REST APIs.
The Dev Tools console allows you to:
- Send HTTP requests directly to Elasticsearch.
- View responses in a structured and readable format.
- Experiment safely without writing any application code.
Once you open Kibana, navigate to Dev Tools → Console. This is where we will run all our commands.
Viewing All Existing Indices
Before creating anything new, it is always a good practice to check what already exists in the cluster.
To list all indices, we use the following endpoint:
GET _cat/indices
However, when you run this command for the first time, you might notice that the output does not contain column headers. This can make it slightly difficult to understand what each column represents.
To fix this, Elasticsearch provides a simple query parameter called v, which stands for verbose. When we include this parameter, Elasticsearch displays column headers along with the data.
GET _cat/indices?v
Once you execute this command, you should be able to clearly see details such as index name, health status, number of shards, and storage size.
Sample Output:
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size dataset.size green open .internal.alerts-transform.health.alerts-default-000001 G-UdPRAgQnybCiOi-878cA 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.logs.alerts-default-000001 gAz0duPOQnKVtLVf2iKzaQ 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.uptime.alerts-default-000001 QEgZcEn9TGeFDVb_M3LAxw 1 0 0 0 249b 249b 249b green open .internal.alerts-ml.anomaly-detection.alerts-default-000001 9rAk9AE0QNSZoNoNPEuKCw 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.slo.alerts-default-000001 yz3dRwtmSGO96tfEJ5sbLQ 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.apm.alerts-default-000001 q6pKGMaSR6aHTd951lR_jg 1 0 0 0 249b 249b 249b green open .internal.alerts-default.alerts-default-000001 C4VRDLfsQvOvvM_B2vLscg 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.metrics.alerts-default-000001 SP4CSkA2TeeROTeNsZ2Mvg 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.threshold.alerts-default-000001 gpebbZg8RFyZz7uiL220qw 1 0 0 0 249b 249b 249b green open .internal.alerts-ml.anomaly-detection-health.alerts-default-000001 DHD188D4TAWhlfSuNnnAug 1 0 0 0 249b 249b 249b green open .internal.alerts-security.alerts-default-000001 YwEaZQ4URWmFsJcidUHYqA 1 0 0 0 249b 249b 249b green open .internal.alerts-stack.alerts-default-000001 qeOrnJ-rTa-VHHSpJhdgXQ 1 0 0 0 249b 249b 249b
Understanding System Indices
When you view the list of indices, you may notice several entries that you did not create yourself. These are known as system indices.
System indices are created and managed internally by Elasticsearch and Kibana. They store metadata, configuration, and internal state required for the platform to function correctly.
This is similar to how relational databases such as PostgreSQL or MySQL maintain internal system tables that users typically do not modify directly.
You should not delete or modify these system indices, especially while learning, as they are critical for Elasticsearch’s internal operations.
Filtering Indices Using a Pattern
As your system grows, you may end up with many indices. In such cases, finding a specific index becomes inconvenient if you list everything.
Elasticsearch allows you to filter indices using patterns directly in the endpoint.
For example, if you want to list only those indices that contain the word transform, you can use the following command:
GET _cat/indices/*transform*
This tells Elasticsearch to return only those indices whose names match the given pattern.
This feature becomes extremely useful in real-world systems where dozens or even hundreds of indices may exist, and you need a quick way to narrow down your search.
Creating a New Index
Now that we understand how to inspect existing indices, let’s create our own.
Suppose we want to store product-related information such as name, price, and category. For this purpose, we will create an index called products.
To create an index, we use the following request:
PUT /products
Once this request is sent, Elasticsearch will create the index and return a response indicating whether the operation was successful.
If you see a response containing something like:
{
"acknowledged": true
}
it means that Elasticsearch has successfully processed your request and the index has been created.
At this point, the products index exists and is ready to store documents.
Understanding the Response and Shards
When an index is created, Elasticsearch internally assigns shards to it. Shards are essentially smaller pieces of an index that allow Elasticsearch to distribute data and scale efficiently.
At this stage, you do not need to worry about shard configuration. We will cover shards in detail in upcoming sections, where their purpose and behavior will become much clearer.
For now, it is enough to understand that Elasticsearch automatically handles this for you.
A Note on Elasticsearch API Design
You might notice something unusual if you come from a strict REST background.
For example, creating a resource often uses the POST method in RESTful systems, but Elasticsearch allows (and often prefers) the use of PUT for creating indices.
This design choice has received mixed reactions from developers over time. However, the Elasticsearch team has clearly stated that while their APIs are REST-like, they do not strictly follow REST conventions.
This behavior is intentional, and once you work with Elasticsearch for a while, it will start to feel natural. It is best to accept this design choice rather than fight it.
Checking Cluster Health After Creating an Index
After creating an index, it is a good idea to check the overall health of the cluster.
You can do this using the following request:
GET _cluster/health
At this point, you may notice that the cluster health status is yellow instead of green.
This often surprises beginners, but there is no need to panic.
Sample Output:
{
"cluster_name": "docker-cluster",
"status": "yellow",
"timed_out": false,
"number_of_nodes": 1,
"number_of_data_nodes": 1,
"active_primary_shards": 33,
"active_shards": 33,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 1,
"unassigned_primary_shards": 0,
"delayed_unassigned_shards": 0,
"number_of_pending_tasks": 0,
"number_of_in_flight_fetch": 0,
"task_max_waiting_in_queue_millis": 0,
"active_shards_percent_as_number": 97.05882352941177
}
Why the Cluster Health Is Yellow
A yellow cluster status simply means that some replicas are not allocated. This usually happens when you are running Elasticsearch on a single-node setup, which is very common during local development.
Elasticsearch is designed to be distributed across multiple nodes. When you create an index, it expects replicas to exist on different nodes for fault tolerance. Since your local setup has only one node, Elasticsearch cannot assign replicas, and therefore shows a yellow status.
This does not mean your system is broken.
It simply means that your cluster is not highly available, which is completely acceptable for learning and development purposes.
In upcoming sections, when we work with multiple nodes, this behavior will make much more sense.
Deleting an Index
Deleting an index is straightforward and uses a simple DELETE request.
If you want to remove the products index, you can run:
DELETE /products
Once this request is executed, the index is permanently removed from the cluster.
After deletion, if you check the cluster health again, it should return to green, assuming no other issues exist.