The field timestamp doesn't exist #9

ZulfuqarAliyev · 2018-07-18T08:51:55Z

Hi. I try to use your appender. But when I creat index in Kibana there is not filter by timestamp and my index doesn't have the field "timestamp". Could you please help me. Thanks in advance

rfoltyns · 2018-07-18T22:33:56Z

It looks like the index mappings are missing.

There's no timestamp field in serialized LogEvent - there's timeMillis instead. You can specify timeMillis mapping via IndexTemplate tag.

It seems that the usage example from the main page is a bit too simple. I'll update the documentation to try to prevent these kind of issues.

UPDATE: Since 2.11.0, log4j-core:org.apache.logging.log4j.core.jackson.LogEventJsonMixIn ignores LogEvent.getTimeMillis. Given that, you have a couple of options here:

switch to log4j-core <= 2.10.0
provide your own JsonLayout that doesn't ignore this getter
specify messageOnly="true" at appender level and start logging your own objects that you fully control (see user-provided format example)

In the meantime, I'll start work on a workaround for this.

kajahno · 2018-08-14T17:53:02Z

In my case something I can notice is that Kibana won't recognize the field timeMillis as a date field, because in the default template this field is coming as a long (see this one):

{
  "mapping": {
    "index": {
      "properties": {
[...]
        },
        "timeMillis": {
          "type": "long"
        }
[...]

My very basic configuration in log4j2.xml is the following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xml>
<configuration>
	<appenders>
		<Elasticsearch name="elasticsearchAsyncBatch">
			<RollingIndexName indexName="log4j2" pattern="yyyy-MM-dd" />
			<AsyncBatchDelivery>			
				<JestHttp serverUris="http://[HIDING MY SERVER IP]:9200" />
			</AsyncBatchDelivery>
		</Elasticsearch>
	</appenders>
	<loggers>
		<root level="debug">
			<appender-ref ref="elasticsearchAsyncBatch" />
		</root>
	</loggers>
</configuration>

And I'm testing it with the following very basic Java code (I don't really know almost anything about coding in Java):

package org.elasticsearch.elastictest;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

/**
 * Hello world!
 *
 */
public class App 
{
    private static Logger logger = LogManager.getLogger(App.class);
	
    public static void main( String[] args ) throws InterruptedException
    {
        System.out.println("Before");
        logger.info("Some informative log line. ZzZzZ");
        logger.warn("Now adding some warnings");
        logger.error("Exception",new Exception("Unexpected DB exception. Some data missing in the database. Tell somebody and go to the pub in the meantime"));
        System.out.println("After");
        Thread.sleep(2000);
        System.exit(0);
    }
}

Temporary Solution
So, one way I managed to fix this is to create the index in elasticsearch before sending the log data making sure the field timeMillis is a date (with the command curl -H 'Content-Type: application/json' -XPUT http://localhost:9200/log4j2-2018-08-14?pretty -d @mymapping.json ). The content of "mymapping.json" is the following:
log4jtemplate.json.txt

Something I'd love is if by default the library would send this field as a date rather than a long, so Kibana is able to interpret it as a timestamp (without doing much voodoo).

Saying that, I tried to change my log4j2 configuration so it will load the template file in the appender itself, which looks very very similar to the example provided here:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xml>
<configuration>

	<appenders>

		<Elasticsearch name="elasticsearchAsyncBatch">
			<RollingIndexName indexName="log4j2" pattern="yyyy-MM-dd" />
			<AsyncBatchDelivery>			
				<IndexTemplate name="log4j2_template" path="classpath:log4jtemplate.json" />
				<JestHttp serverUris="http://[HIDING MY SERVER IP]:9200" />
			</AsyncBatchDelivery>
		</Elasticsearch>
        <Async name="asyncLogger">
            <AppenderRef ref="elasticsearchAsyncBatch" />
        </Async>
	</appenders>

	<loggers>
        <AsyncLogger name="elasticsearch" level="info" additivity="false">
            <AppenderRef ref="asyncLogger" />
        </AsyncLogger>
        <Root level="info" />
	</loggers>

</configuration>

But the only thing I get back is:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2018-08-14 17:52:42,349 main ERROR Could not create plugin of type class org.appenders.log4j2.elasticsearch.AsyncBatchDelivery for element AsyncBatchDelivery org.apache.logging.log4j.core.config.ConfigurationException: IndexTemplate not added: {"root_cause":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: index patterns are missing;"}],"type":"action_request_validation_exception","reason":"Validation Failed: 1: index patterns are missing;"}
	at org.appenders.log4j2.elasticsearch.jest.JestHttpObjectFactory.execute(JestHttpObjectFactory.java:154)
	at org.appenders.log4j2.elasticsearch.AsyncBatchDelivery.<init>(AsyncBatchDelivery.java:62)
	at org.appenders.log4j2.elasticsearch.AsyncBatchDelivery$Builder.build(AsyncBatchDelivery.java:147)
	at org.appenders.log4j2.elasticsearch.AsyncBatchDelivery$Builder.build(AsyncBatchDelivery.java:100)
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:122)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:958)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:898)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:890)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:890)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:513)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:237)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:249)
	at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:545)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:617)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:634)
	at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:229)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:152)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
	at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
	at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:551)
	at org.elasticsearch.elastictest.App.<clinit>(App.java:15)

2018-08-14 17:52:42,356 main ERROR No BatchDelivery method provided for ElasticSearch appender: batchDelivery
2018-08-14 17:52:42,356 main ERROR Could not create plugin of type class org.appenders.log4j2.elasticsearch.ElasticsearchAppender for element Elasticsearch org.apache.logging.log4j.core.config.ConfigurationException: Arguments given for element Elasticsearch are invalid
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.injectFields(PluginBuilder.java:203)
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:121)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:958)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:898)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:890)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:513)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:237)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:249)
	at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:545)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:617)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:634)
	at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:229)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:152)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
	at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
	at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:551)
	at org.elasticsearch.elastictest.App.<clinit>(App.java:15)

2018-08-14 17:52:42,370 main ERROR Null object returned for Elasticsearch in appenders.
2018-08-14 17:52:42,378 main ERROR Unable to invoke factory method in class org.apache.logging.log4j.core.async.AsyncLoggerConfig for element AsyncLogger: java.lang.NoClassDefFoundError: com/lmax/disruptor/EventFactory java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:136)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:958)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:898)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:890)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:513)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:237)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:249)
	at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:545)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:617)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:634)
	at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:229)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:152)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
	at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
	at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:551)
	at org.elasticsearch.elastictest.App.<clinit>(App.java:15)
Caused by: java.lang.NoClassDefFoundError: com/lmax/disruptor/EventFactory
	at org.apache.logging.log4j.core.config.AbstractConfiguration.getAsyncLoggerConfigDelegate(AbstractConfiguration.java:202)
	at org.apache.logging.log4j.core.async.AsyncLoggerConfig.<init>(AsyncLoggerConfig.java:82)
	at org.apache.logging.log4j.core.async.AsyncLoggerConfig.createLogger(AsyncLoggerConfig.java:189)
	... 20 more
Caused by: java.lang.ClassNotFoundException: com.lmax.disruptor.EventFactory
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 23 more

2018-08-14 17:52:42,387 main ERROR Null object returned for AsyncLogger in loggers.
2018-08-14 17:52:42,388 main ERROR No appender named elasticsearchAsyncBatch was configured
Exception in thread "main" java.lang.ExceptionInInitializerError
Caused by: org.apache.logging.log4j.core.config.ConfigurationException: No appenders are available for AsyncAppender asyncLogger
	at org.apache.logging.log4j.core.appender.AsyncAppender.start(AsyncAppender.java:120)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:265)
	at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:545)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:617)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:634)
	at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:229)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:152)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
	at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
	at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:551)
	at org.elasticsearch.elastictest.App.<clinit>(App.java:15)

Again, maybe I'm doing something bad (my knowledge of Java is pretty bad), so it'll be really nice if you could point me in the right direction for this one.

In any case I'd appreciate any sort of feedback.

Cheers!

rfoltyns · 2018-08-14T19:02:33Z

Your mappings are correct, but there are two issues in these stacktraces:

"reason":"Validation Failed: 1: index patterns are missing;" Lack of index pattern in log4jtemplate.json. You can fix it by adding template field to your JSON file:
{ "template": "log4j2*", mappings: ... }
java.lang.ClassNotFoundException: com.lmax.disruptor.EventFactory com.lmax:disruptor.jar was not provided on your classpath. You can fix it by adding this dependency to your pom.xml (give it a go, it's worth it) or (not recommended in high throughput scenarios) switching to synchronous logger:
<Logger name="elasticsearch" level="info" additivity="false"> <AppenderRef ref="elasticsearchAsyncBatch" /> </Logger>

kajahno · 2018-08-15T09:21:04Z

Cheers @rfoltyns , works fine now!

…pgraded * layout implements ItemSourceLayout * layout allows to output customisations via FasterXML Jackson mixins * LogEventJacksonJsonMixIn (modified log4j-core:LogEventJsonMixIn) was added to prevent #9 (visit javadoc for full list of changes) * this layout is now used by default

* IndexTemplate should be added to prevent #9

rfoltyns · 2019-01-08T20:30:27Z

@ZulfuqarAliyev @kajahno Log4j2 version compatibility issues were addressed in release 1.3.0. JacksonJsonLayout uses a set of patched MixIns, so timeMillis should now be serialized OOTB.

Could you retest, please?

rfoltyns · 2019-02-21T16:42:45Z

Fixed in 1.3.0

rfoltyns added a commit that referenced this issue Dec 20, 2018

Documentation - IndexTemplate added to config snippets

0a25106

* IndexTemplate should be added to prevent #9

rfoltyns pinned this issue Jan 8, 2019

rfoltyns closed this as completed Feb 21, 2019

rfoltyns added the howto label May 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The field timestamp doesn't exist #9

The field timestamp doesn't exist #9

ZulfuqarAliyev commented Jul 18, 2018

rfoltyns commented Jul 18, 2018 •

edited

Loading

kajahno commented Aug 14, 2018 •

edited

Loading

rfoltyns commented Aug 14, 2018

kajahno commented Aug 15, 2018

rfoltyns commented Jan 8, 2019

rfoltyns commented Feb 21, 2019

The field timestamp doesn't exist #9

The field timestamp doesn't exist #9

Comments

ZulfuqarAliyev commented Jul 18, 2018

rfoltyns commented Jul 18, 2018 • edited Loading

kajahno commented Aug 14, 2018 • edited Loading

rfoltyns commented Aug 14, 2018

kajahno commented Aug 15, 2018

rfoltyns commented Jan 8, 2019

rfoltyns commented Feb 21, 2019

rfoltyns commented Jul 18, 2018 •

edited

Loading

kajahno commented Aug 14, 2018 •

edited

Loading