python - Using downloaded NLTK data on AWS Elastic Beanstalk -


i have django app running on aws elastic beanstalk. use nltk corpus package (stopwords) obtain using nltk downloader.

for quick hack, ran nltk downloader on current (single) elastic beanstalk ec2 instance , saved needed corpus /usr/local/share/nltk_data. works on single instance when load balancer decides create new instances wiped (it survives deployments).

my question best approach here data?

should store on s3 , tie elastic beanstalk?

or, easier (and better) write (python?) script called eb configuration each new instance download , place data in folder accessible app (for life of instance)? way if need add other corpus downloads or python-specific or nltk-specific things it's happening in python , not requiring manual s3 work?

if supports writing script eb configuration, example great, not sure how exactly.

thanks!

it easy use s3 specific use-case (in combination iam , ec2 instance roles).

even fast changing data (nltk corpora slow changing assume), 1 can manually sync differences existing s3 location instances have new data available when need it.

the key give instances iam roles, using instance profiles. proper policy, can securely access s3 without having define aws credentials manually, or in script needs access aws cli on instance start, etc.

there significant security benefits using instance profiles iam permissions aws resources, eliminates hard-coding credentials scripts, git code etc.

then assuming aws cli installed on linux via apt, pip etc:

 # create bucket (once).   # put in region / az ec2 instances   # minimize data xfer   # can run these wherever bucket / data  aws s3 mb s3://mybucket --region us-west-1   # sync wherever first time & whenever needed  aws s3 sync /usr/local/share/nltk_data s3://mybucket    # can run below on instances  #  # put instance startup script after install of awscli etc.  # or in myscript.sh file on instance (even gist)  # wherever want instance have data or sync   aws s3 sync s3://mybucket/nltk_data /path/where/i/need

the nice thing sync command not copy on files have not been modified when putting s3 , pulling down. makes super handy things common datasets, backups, etc.


Comments

Post a Comment

Popular posts from this blog

sql server - Cannot query correctly (MSSQL - PHP - JSON) -

i have little problem on project. , turns out query result returns nothing when value of first name or last name has 'ñ' in it. here's code config.php: <?php ini_set('mssql.charset', 'utf-8'); header('content-type: text/html; charset=utf-8'); $servername = "192.168.1.21"; /* ip add of db server */ $connection = array("database" => "db", "uid" => "?", "pwd" => "***"); $conn = sqlsrv_connect($servername, $connection); if(!$conn){ echo "connection not established."; die(print_r(sqlsrv_errors(), true)); } ?> myquery: $idnumber = $_post["idnum"]; $response = array(); $query2 = "select clname, cfname, cmname, cqualifier student cidno = '$idnumber'"; try{ $stmt2 = sqlsrv_query($conn, $query2); } catch(pdoexeption $ex){ $response["success"] = 0; $response["message"] = "d...
Read more

php - trouble displaying mysqli database results in correct order -

i have table in database stores events , date expire. want display events on website display current events first expired events after? i have tried following sql $sql = "select noticeid, notice, noticedesc, noticeimg, noticedate, expirydate tbl_notices group noticeid ,expired order expirydate asc"; but returns expired results first due expirydate being ordered asc in query. my tbl_notice structure looks this noticeid => int(4) notice => varchar(100) noticedesc => text noticedate => timestamp noticeimg => varchar(40) expirydate => datetime expired => int(1) 0 current or 1 expired my php code is $sql = "select noticeid, notice, noticedesc, noticeimg, noticedate, expirydate tbl_notices group noticeid ,expired order expirydate asc"; $result = mysqli_query($conn,$sql); if ($result->num_rows > 0) { // output data of each row while($row = $result->fetch_assoc()) { ...
Read more

C++ Linked List -

i trying create linked list download txt file , use linked list handle file line line. when handling downloaded linked list operations performed on such text editor would. encountering problems however. seems "node(string value)" section of code has wrong though original node() declaration no arguments passes. unable figure out quite is. node.h class node { public: node(); node(string value); void setnext(node *nextnode); // allows user set "next" pointer of node points friend class linkedlist; private: string data; // data box node* next; // pointer box }; node.cpp # include <string> # include "node.h" using namespace std; node::node() { data = ""; next = null; } node::node(string value) { data = value; next = null; } void node::setnext(node *nextnode) // allows user set "next" pointer of node points { this->next = nextnode; } your #include <string> ...
Read more