java - JDBC driver cannot be found when reading a DataSet from an SQL database in Apache Flink -


after having followed beginner java tutorials apache flink on documentation sites wanted try transformations on own data. however, i'm having trouble gathering input microsoft sql database running on server in network.

the examples in section possible sources datasets contain section looked need, dataset built using env.createinput(...) jdbcinputformat. added maven dependency flink jdbc

<dependency>     <groupid>org.apache.flink</groupid>     <artifactid>flink-jdbc_2.11</artifactid>     <version>0.10.2</version> </dependency>

and remodeled given code fit own database this:

// create , configure input format jdbcinputformat inputformat = jdbcinputformat.buildjdbcinputformat()     .setdrivername("org.apache.derby.jdbc.embeddeddriver")     .setdburl(sqlserver)     .setusername(username)     .setpassword(password)     .setquery(query)     .finish();  // create , configure type information dataset tupletypeinfo typeinformation = new tupletypeinfo(tuple2.class, string_type_info, int_type_info);  // read data relational database using jdbc input format dataset<tuple2<string, integer>> dbdata = environment.createinput(inputformat, typeinformation);

server address, user name , password same work in java program of mine use jdbc only. query simple select on 2 columns, 1 containing string values, other integers.

when running program classnotfoundexception referring selected driver: jdbc-class not found. - org.apache.derby.jdbc.embeddeddriver @ org.apache.flink.api.java.io.jdbc.jdbcinputformat.open

now, seem missing imports here, can't figure out (and them), expecting flink jdbc support minimal example. same driver name given in jdbcinputformat javadoc. tried adding jdbc 4.2 manually did not work.

what need add or change driver found? additionally, there official material flink jdbc , usage, apart javadoc? having difficulties finding tutorials flink , sql sources in general.

  1. if want read data microsoft sql server database, should use jdbc driver sql server, not 1 apache derby. jdbc drivers included in dbms distribution / installation. maybe microsoft offers corresponding jar file download on website.

  2. the driver must added classpath. there 2 options: 1) bundle in application jar, i.e., add include in fat jar or 2) add apache flink's ./lib folder (note, must added flink installations of cluster.


Comments

Post a Comment

Popular posts from this blog

asynchronous - C# WinSCP .NET assembly: How to upload multiple files asynchronously -

i using winscp .net assembly uploading files. want upload multiple files asynchronous way. have created method works single upload. public class uploadingdata { private sessionoptions _sessionoptions; private session _session; //connection etc private void onconnected() { foreach (var fileinfo in localfileslist) { var task = task.factory.startnew(() => uploadfilesasync(fileinfo)); } } private async task uploadfilesasync(string file) { string remotefilepath = _session.translatelocalpathtoremote(file, @"/", "/test_data_upload"); var uploading = _session.putfiles(file, remotefilepath, false); //when done await task.run(() => thread.sleep(1000)); } } please suggest me correct way. thanks you can use backgroundworker class this: string[] images = new string[50]; foreach (var path in images) { backgroundworker
Read more

aws api gateway - SerializationException in posting new Records via Dynamodb Proxy Service in API -

i getting "__type": "com.amazon.coral.service#serializationexception" as reply in postman & in test console in api gateway trying post record directly dynamodb using api proxy services.. referring aws article - https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/ here's mapping { "tablename": "tablenamegoeshere", "item": { "id" : "$context.requestid" "eventname" : "$input.path('$.eventname')", "timestamp" : $input.path('$.timestamp'), "answers": "$util.parsejson($input.path('$.answers'))" } } update: did asked ... , worked if try add array of json objects gives me above same error - here's trying now. please - couldnt find on google well #set($inputroot = $input.path('$')) { "tablename": "answer", "item": {
Read more

asp.net - Problems sending emails from forum -

i have been redesigning site has message board, has been written guy looked after site. it has few problems , written in .asp , uses .mdb files database. you can see whole thing here . feel free post if helps, test thing have set using original files , new database. the basic problem when posts message, supposed email sent them, have posted it. when replies, originator supposed message, replier has been posted. i’d have thought @ point in process there should script/file in postmessage.asp? the postmessage.asp page below. <% option explicit %> <!-- #include virtual="common/adovbs.inc" --> <html> <head> <meta http-equiv="content-type" content="text/html; charset=windows-1252"> <meta name="generator" content="microsoft frontpage 4.0"> <meta name="progid" content="frontpage.editor.document"> <title>new page 2</title> <base target="_self"
Read more