Report: 35% year-over-year increase in logging data burdens software engineers

Managing logs has become increasingly critical for every business in every industry. New Relic observed a 35% YoY increase in logging data.

As the volume of log files grows, software engineers want to access log data available in one place to speed up the time to detect and respond to transactions, errors and security incidents. Centralized log management was created by engineers frustrated by the need to examine thousands of log files across a number of sources to pinpoint and resolve incidents.

Logging data processors

According to New Relic’s 2022 State of Logs report, which examined logging data gathered from
millions of applications within the New Relic observability platform, Fluent Bit is the most used open-source processor and forwarder tool amongst New Relic users (38% use it). It has a massive following because of its ability to manage observability data at scale across cloud-native, internet of things, and bare metal environments.

Next up is the New Relic infrastructure agent (16%), followed by sending log data directly to New Relic (14%) via an HTTP endpoint.

NGINX is the most popular type of log, capturing 38%, followed by Syslog (25%), ALB (20%), and Microsoft IIS W3C (9%). Web servers, load balancers and content delivery networks (CDNs) are often used by engineers to perform wellness checks on applications. They are not easily monitored with agents, which is why these three are the most common type of log sent to New Relic.

Log file forwarders and languages

Firehose will soon be the de facto log forwarder for AWS serverless users. Amazon Lambda is currently in the top spot, with 46% of all AWS users in New Relic’s customer pool adopting the technology, but the growth rate of users adopting Amazon Firehose (62% YoY) is staggering and New Relic anticipates Firehose taking the lead later this year or in early 2023. The rise in popularity can be attributed to a significant investment in integrations, with broad availability across its partner ecosystem.

Among languages, Java has a commanding lead with the most application log data. The
data shows that 50% of all logs ingested by language agents comes from Java, followed by
.Net (26%), Ruby (22%), and Node.js (2%). The Java lead aligns with the overall popularity of
Java with software developers and the adoption of the Apache Log4j.

Read the full report from New Relic.

Originally appeared on: TheSpuzz