We stopped using it for analytics because of its price, and at the moment, we are using it mostly for log centralization. If you use it with high traffic for analytical purposes, as well as for the logs, the infrastructure costs are unbelievable.

Graylog is a great product backed by Elasticsearch as the storage and query engine. It is just an interface on top of Elasticsearch and some Elasticsearch management. The indexes that are kept in Elasticsearch are managed by Graylog software. Elasticsearch is a decent product, but it's very infrastructure-heavy. It requires lots of resources, and if you make a mistake with provisioning, you are likely to not get a cluster back. We had a couple of outages like that, and we hated that. So, we ended up over-provisioning resources just to avoid such situations from happening. If you have a whole team trying to fix the Graylog instance for two days, that's a bit too much. That may be my Norwegian take on it, but the engineering resources are expensive. It's better to just provision the infrastructure.

Overall, the product is great, and the features are just fine, but the infrastructure cost is what is killing it. The infrastructure cost is the main issue. I like the rest. If the infrastructure costs could be lower, it would be fantastic. I'm not sure if they can improve the infrastructure cost with the way Elasticsearch is. If they keep using Elasticsearch, maybe there are some opportunities there, or they can support other backends with cheaper storage. They could have a different backend to replace Elasticsearch or do some tweaks to Elasticsearch to reduce the costs. There could be partial parsing of logs or parsing on demand so that when you write data through Graylog to Elasticsearch, it doesn't need to crunch in every detail requiring that much CPU.

Graylog needs to improve their authentication. Their AD integration is really bad. When it comes to ACL's, access control lists, where you want to have different group memberships and control who gets access to what, it really could use major improvements. It seems like a beta authentication version that they came up with in a hurry and said, "Hey guys, we've got something going for you. Use it until we think of something later on." I believe their enterprise version has improved some of these features, but I use the open source version.

The second thing would be the way they handle live logging. The fact that Graylog displays logs from the top down is just ridiculous. I've never seen anything that logs this way except for Graylog. I believe this is an issue because they have the selector going in that direction, so it would make sense that they have to implement it that way, but it's definitely not cool. When you're looking at Graylog's live logging, whether it's doing a one-second or five-second pull, you'll notice that new log lines are placed at the top of the screen, not the bottom of the screen. I find this ridiculous because normally when you're looking at logs anywhere, on Linux, even in Windows, you're going to see that the logs are generated at the bottom. That's one thing that Graylog definitely needs to improve. 

Graylog also needs to invest some time to improve the performance and how they handle the maintenance of Elasticsearch.

An added feature I would like to see is the capacity to delegate most of the backend maintenance to the frontend UI. When you have somebody from the service desk working on the solution or somebody who's not a technical person, they could run some of the maintenance stuff directly from the UI.

Another thing is something that I saw in LogDNA, where you could have a color based on log regex. For example, it would color the timestamps next to the log lines orange, make the source of the log purple, and then make the actual log content black. That would be very nice to see in Graylog.

With Python, there was a problem where it was harder to attach extra information using the basic logging package. We had to build our own custom adapter for this to append that information to the log message. For Python developers, it would be great if Graylog could provide a better Python package in order to make it easier to use for the Python community.

Since it's a free tool, I don't have much to say. Troubleshooting is important to me. The initial setup is complex. I hope to see improvements in Graylog for more interactivity, user-friendliness, and creating alerts.

Graylog could improve the process of creating rules. We have to create them manually by doing parses and applying them. Other SIEM solutions have basic rules and you can create and get more events of interest.

Elasticsearch recommendations for tuning could be better. Graylog doesn't have direct support for running the system inside of Kubernetes, so it can be challenging to fill in the gaps and set up containers in a way that is both performant and stable.

We ran into problems with Elasticsearch throwing a circuit-breaking exception due to field data size being too large. It turned out that the heap size directly impacted this size in a high-throughput environment, causing unexplained instability in Graylog. We were able to troubleshoot on the Elasticsearch size, but we should have been able to reference some minimum requirements for Graylog to know that our settings weren't sufficient.

Otherwise, the documentation is great and there are a lot of options for configuration. Since container orchestration systems are popular and Graylog fits the niche well, perhaps they could officially support running in docker containers on Kubernetes as a StatefulSet as a use case. That way, the declarative nature of Kubernetes config files would document their best-case deployment scenario.

Graylog can improve the index rotation as it's quite complicated. They need to work on that because it's quite cumbersome to manage the index rotation with all the logs.

The filtering of logs before ingestion also needs a bit of work. This is because you have to write some code to avoid certain things before ingesting. As it doesn't support certain AIX versions, you need to upgrade the servers to accommodate it.

They depleted the legacy alarm callback feature from the current version. They should make it available in the newest version as well. Also, they should include SSO integration in Graylog 5.0's community version, similar to its enterprise version. It would also be beneficial if they added a feature that scales the solution automatically when the load increases.

The collectors and using sidecar made my life easier from earlier versions. Unfortunately, I have been pulled away from the product, beyond setting up new inputs, defining the alerts. I am currently trying to leverage the API and Graylog Extended Log Format (GELF), and some of the underlying tech of Elasticsearch as well, for downstream consumers and our AI consumers.

For improvements or features to add, I would like to see a default dashboard widget that shows the topology of the clusters defined for the graylog install.
For instance, I have three Elasticsearch nodes and three MongoDB. I would like to see a visual representation of their status. 

Additionally, maybe it does exist (I have not looked), but I would like to see percent filled of the current index. 

• Backup and restore functionally for migrating instances.
• Dashboard and search analytics (i.e., more complex visualizations and the ability to execute custom Elasticsearch queries would be great).
• More flexible alert conditions

I would like to see a date and time in the Graylog Grok patterns so that I can save time when searching for a log. I like how the streams and the search query work, but adding a date and time will allow me to pull out a log in a milli-second.

The biggest problem is the collector application, as we wanted to avoid using Graylog Collector Sidecar due to its architecture. It requires connection outside our network during build from source, so we decided instead to use the obsolete Graylog Collector, which is working fine and in an easy way. It would be great, if that component would get back into the development process. But it is nothing that I could even complain about, as our company is not paying for support.

I would like to see some kind of visualization included in Graylog. The report is plain, they could be improved.

The alerting system could be more flexible. It does not allow for definition of different thresholds and alert types of the same streams. It allows different alert types and thresholds for the same stream.

E.g., if we have a single stream of errors, I would like to send each error to the ticketing system: A mail if there are less than 1 errors per second and an SMS if greater than 10 errors received per second.

There are places which could be improved:

• Stream alerts
• Dashboards
• Parsing.

Some places were already improved in 2.4 with the threat intelligence add-on.

There should be some user groups and an auto sign-in feature.

It would be helpful if they would work more on the documentation because it's not very clear and ideally I'd like to be able to do more myself, but would need some additional guidelines and material for that.

More customization is always useful.