Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot add extension druid-datasketches ? #309

Open
longhopefor opened this issue Oct 14, 2019 · 3 comments
Open

cannot add extension druid-datasketches ? #309

longhopefor opened this issue Oct 14, 2019 · 3 comments

Comments

@longhopefor
Copy link

longhopefor commented Oct 14, 2019

tranquility 0.8.2
druid 0.16.0

run:
bin/tranquility server -configFile conf/server.json -Ddruid.extensions.loadList='["druid-datasketches"]' -Ddruid.extensions.directory=/usr/local/druid-0.16.0/extensions

in server.json, i add:

"metricsSpec": [
{
"name" : "count",
"type" : "count"
},
{
"name":"sketch_uid",
"type":"thetaSketch",
"fieldName":"uid",
"isInputThetaSketch":"false",
"size":"1048576"
}
]

output:
2019-10-14 09:19:46,634 [main] INFO i.d.initialization.Initialization - Loading extension [druid-datasketches] for class [io.druid.initialization.DruidModule]
2019-10-14 09:19:46,701 [main] INFO i.d.initialization.Initialization - added URL[file:/usr/local/druid-0.16.0/extensions/druid-datasketches/commons-math3-3.6.1.jar]
2019-10-14 09:19:46,701 [main] INFO i.d.initialization.Initialization - added URL[file:/usr/local/druid-0.16.0/extensions/druid-datasketches/druid-datasketches-0.16.0-incubating.jar]
2019-10-14 09:19:46,701 [main] INFO i.d.initialization.Initialization - added URL[file:/usr/local/druid-0.16.0/extensions/druid-datasketches/memory-0.12.2.jar]
2019-10-14 09:19:46,702 [main] INFO i.d.initialization.Initialization - added URL[file:/usr/local/druid-0.16.0/extensions/druid-datasketches/sketches-core-0.13.4.jar]
2019-10-14 09:19:46,702 [main] INFO i.d.initialization.Initialization - added URL[file:/usr/local/druid-0.16.0/extensions/druid-datasketches/slf4j-api-1.7.25.jar]
2019-10-14 09:19:47,070 [main] INFO c.metamx.emitter.core.LoggingEmitter - Start: started [true]
2019-10-14 09:19:47,545 [main] WARN io.druid.segment.indexing.DataSchema - No metricsSpec has been specified. Are you sure this is what you want?
2019-10-14 09:19:47,547 [main] WARN io.druid.segment.indexing.DataSchema - No metricsSpec has been specified. Are you sure this is what you want?
2019-10-14 09:19:47,550 [main] INFO c.metamx.emitter.core.LoggingEmitter - Start: started [true]
2019-10-14 09:19:47,552 [main] WARN io.druid.segment.indexing.DataSchema - No metricsSpec has been specified. Are you sure this is what you want?
java.lang.IllegalArgumentException: Could not resolve type id 'thetaSketch' into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]
at [Source: N/A; line: -1, column: -1] (through reference chain: Object[][1])
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2774)
at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700)
at com.metamx.tranquility.druid.DruidBeams$.makeFireDepartment(DruidBeams.scala:433)
at com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:299)
at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:204)
at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:118)
at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:98)
at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)
at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.immutable.Map$Map3.foreach(Map.scala:161)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at com.metamx.common.scala.collection.package$MapLikeOps.strictMapValues(package.scala:143)
at com.metamx.tranquility.server.http.ServerMain$.createServlet(ServerMain.scala:98)
at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:168)
at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:167)
at scala.Option.foreach(Option.scala:257)
at com.twitter.app.App$class.nonExitingMain(App.scala:167)
at com.metamx.tranquility.server.http.ServerMain$.nonExitingMain(ServerMain.scala:49)
at com.twitter.app.App$class.main(App.scala:133)
at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:49)
at com.metamx.tranquility.distribution.DistributionMain$.main(DistributionMain.scala:34)
at com.metamx.tranquility.distribution.DistributionMain.main(DistributionMain.scala)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id 'thetaSketch' into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]
at [Source: N/A; line: -1, column: -1] (through reference chain: Object[][1])
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:862)
"format": "json"
at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:167)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)
at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:17)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:124)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:124)
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2769)

@partsalliance
Copy link

I am having the same issue with quantilesDoublesSketch.

@indrekj
Copy link

indrekj commented May 28, 2020

Same problem here, @longhopefor @partsalliance were you able to solve it?

From the logs it seems that the extension itself is loaded:

May 28 09:11:48 ip-10-4-0-31 tranquility[24856]: 2020-05-28 09:11:48,904 [main] INFO  io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=tr
ue, directory='/usr/local/druid/extensions', hadoopDependenciesDir='hadoop-dependencies', hadoopContainerDruidClasspath='null', loadList=[druid-datasketches]}]
May 28 09:11:48 ip-10-4-0-31 tranquility[24856]: 2020-05-28 09:11:48,930 [main] INFO  i.d.initialization.Initialization - Loading extension [druid-datasketches] for class [io.druid.initialization.DruidModule]
May 28 09:11:48 ip-10-4-0-31 tranquility[24856]: 2020-05-28 09:11:48,941 [main] INFO  i.d.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-datasketches/druid-datasketches-0.18.1.jar]
May 28 09:11:48 ip-10-4-0-31 tranquility[24856]: 2020-05-28 09:11:48,941 [main] INFO  i.d.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-datasketches/commons-math3-3.6.1.jar]

@longhopefor
Copy link
Author

i don't use it now, i guess the tranquility is not compatible with the latest druid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants