scala - scalding with Custom Tap could not build flow from assembly -
everyone have scalding project save data hbase using latest or stable engine in environtment, these engine , dependencies used project :
engine :
- zookeeper 3.4.9
- hbase 1.2.3
- hadoop 2.7.2
dependencies :
librarydependencies ++= seq(
"org.apache.hbase" % "hbase" % "1.2.3",
"org.apache.hbase" % "hbase-server" % "1.2.3",
"org.apache.hbase" % "hbase-protocol" % "1.2.3",
"org.apache.hbase" % "hbase-common" % "1.2.3",
"org.apache.hbase" % "hbase-client" % "1.2.3",
"org.apache.hadoop" % "hadoop-core" % "1.2.1",
"org.apache.hadoop" % "hadoop-common" % "2.7.2",
"cascading" % "cascading-core" % "2.6.1",
"cascading" % "cascading-local" % "2.6.1",
"cascading" % "cascading-hadoop" % "2.6.1",
"com.twitter" %% "scalding-args" % "0.15.0",
"com.twitter" %% "scalding-core" % "0.15.0",
"com.twitter" %% "scalding-date" % "0.15.0"
)
and because hbasetap , hbasesource hbase 1.2.3 not available yet, create own hbasetap , hbasesource using reference :
docs.cascading.org/cascading/2.6/userguide/pdf/userguide.pdf
and
[https://github.com/luzhuomi/scalding-hbase][2]
then when run project give me error :
cascading.flow.planner.plannerexception: not build flow assembly: [null]
i run project in hdfs mode code :
yarn jar {path jar} {class} --hdfs --host localhost
the job run in project pretty simple, it's read file , put hbase, :
class hbasetests(args: args) extends job(args) {
val input = textline("/mnt/hello.txt")
val out = new hbasesource("testing", "localhost","stream", 'line ,array('offset))input.read
.write(out) }
and custom hbase tap : mycustomhbasetap.scala
can me fix ? all.
Comments
Post a Comment