Upgrading OpenSearch to Version 2.x from 1.3

Migrating to OpenSearch 2.x
To ensure compatibility with version 2.x, it’s important to reindex older production indexes that may no longer be compatible.

Creating a new production cluster for a safe migration
Create a new cluster with a copy of your production OpenSearch environment, where you’ll perform the complete migration. After testing, you can switch to the new production environment.

Local Testing: Install and Run OpenSearch for Development
Before starting the migration, you may want to test the process locally. Here’s how you can do it:

docker pull public.ecr.aws/opensearchproject/opensearch:latest
docker run -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:latest
curl https://localhost:9200 -ku 'admin:admin'

If you prefer to work without TLS:

• SSH into the Docker container using docker exec -it <container-id> /bin/bash.
• Modify the /usr/share/opensearch/config/opensearch.yml file by setting

plugins.security.disabled: true

Removal of Mapping Types

In OpenSearch 2.x, mapping types have been deprecated. Initially, they were used to draw analogies with SQL databases, which often led to incorrect assumptions. To learn more about this change, read Why are mapping types being removed?

Reindexing

To make your index compatible with 2.x, you’ll need to reindex your previous index. For example:

{
   "source":{
      "index":"live_index"
   },
   "dest":{
      "index":"os_live_index"
   }
}

Comparing Live and Reindexed Indexes

After reindexing, it’s important to compare your live index with the reindexed data:

Creating an Alias

{
  "actions": [
    {
      "add": {
        "index": "os_live_index",
        "alias": "live_index"
      }
    }
  ]
}

Check all aliases

Alternative: Remote reindexing with elasticdump

If you encounter challenges with remote reindexing, consider using Elasticdump as an alternative method elasticdump.

# npm install elasticdump -g
elasticdump \
  --input=https://vpc-os.es.amazonaws.com/live_index \
  --output=http://localhost:9200/live_index \
  --type=analyzer
elasticdump \
  --input=https://vpc-os.es.amazonaws.com/live_index \
  --output=http://localhost:9200/live_index \
  --type=mapping
elasticdump \
  --input=https://vpc-os.es.amazonaws.com/live_index \
  --output=http://localhost:9200/live_index \
  --type=data --limit=500 --concurrency=5

Migration and Refactoring

After completing the reindexing process and patching the upgrades on a separate cluster, it’s time to refactor your code repositories. This involves eliminating the need for mapping type definitions, making interactions more straightforward. Examples of these refactoring changes include:

• Updating GET, DELETE, and POST requests to accommodate the new _doc structure.
• Streamlining filters and bulk operations by removing the type.

// Get single document, replace type with _doc
GET `/live_index/[type]/${id}`  → `live_index/_doc/${id}`

// Delete single document
DELETE `/live_index/[type]/${id}` → `live_index/_doc/${id}` 

// If you're using filters  remove type from filters
`{ "ids": { "type": "mapping_type", "values": [1,2,3,4,5] }}` 
→ 
`{ "ids": { "values": [1,2,3,4,5] }}` 

// Post update without type
POST `/live_index/[type]/${id}/_update` → `/live_index/_update/${id}`

// Refactor bulk update/create etc. requests make sure to remove "type":
BULK `{ "update": { "_index": "live_index", "_id": "123" } }` 
BULK `{ "create": { "_index": "live_index", "_id": "123" } }`

I hope that this guide helps you navigate the migration process to OpenSearch 2.x.