I have encountered an error when trying to create an index, saying “Error forbidden”

turns out the issue is with elastic search.

checking elastic search, there is an error with operation blocked due to “read only”:

this not only blocking the kibana for updating, for example create new index, it blocks new logs from being pumped in as well.

and ultimately, this was set due to the flood_stage watermark:

to sort out the issue, i have cleared some old docker images and containers:

followed by a call to reset the read_only

which brought the elasticsearch back to normal

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store