Showing posts with label amazon ec2. Show all posts
Showing posts with label amazon ec2. Show all posts

Thursday, March 19, 2015

Introduction to Julia

"Julia is a fresh approach to technical computing."  boasts the startup message, flourished with colorful circles hovering above a bubbly ASCII Julia logo.  The formatting effort is not wasted, it's an exuberant promise: Julia will make the command line fun again.
apptrain_1@julia:~/workspace $ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.1 (2014-02-11 06:30 UTC)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |  x86_64-linux-gnu

julia> 

Julia was created by four Data Scientists from MIT who began working on it around 2011.  The language is beginning to mature at a time when the Data Scientist job title is popping up on resumes as fast as Data Scientist jobs appear.  The timing is excellent.   R programming, an offshoot of S Programming , is the language of choice for today's mathematical programmer.  But it feels clunky, like a car from the last century. While Julia may not unseat R in the world of Data Analysis,  plans don't stop there.

If you want to code along with the examples in this article, jump to Getting Started with Julia and chose one of the three options to start coding.

Julia is a general purpose programming language.  It's creators have noble goals.  They want a language that is fast like C, they want it flexible with cool metaprograming capabilities like Ruby, they want parallel and distributed computing like Scala, and true Mathematical equations like MATLAB.



Why program in Julia?

1) Julia is Fast


Julia already boasts faster matrix multiplication and sorting than Go and Java.  It uses the LLVM compiler, which languages like GO use for fast compilation.   Julia uses just in time (JIT) compilation to machine code , and often achieves C like performance numbers.

2) Julia is written in Julia

Contributors need only work with a single language, which makes it easier for Julia users to become core contributors. 
"As a policy, we try to never resort to implementing things in C. This keeps us honest – we have to make Julia fast enough to allow us to do that" -Stephan Karpinski

And, as the languages co-creator Karpinski notes in the comments of the referenced post,   Writing the language itself in Julia means that when improvements are made to the compiler, both the system and user code gets faster.

3) Julia is Powerful

Like most programming languages, it's implementation is Open Source.  Anyone can work on the language or the documentation.  And like most modern programming languages, Julia has extensive metaprogramming support.  It's creators attribute the Lisp language for their inspiration:
Like Lisp, Julia represents its own code as a data structure of the language itself.
a) Optional Strong Typing
Using strong typing can speed up compiling, but Julia keeps strong typing optional, which frees up programmers who want to write dynamic routines that work on multiple types. 
julia> @code_typed(sort(arry))
1-element Array{Any,1}:
 :($(Expr(:lambda, {:v}, {{symbol("#s1939"),symbol("#s1924")},{{:v,Array{Float64,1},0},{symbol("#s1939"),Array{Any,1},18},{symbol("#s1924"),Array{Any,1},18}},{}}, :(begin $(Expr(:line, 358, symbol("sort.jl"), symbol("")))
        #s1939 = (top(ccall))(:jl_alloc_array_1d,$(Expr(:call1, :(top(apply_type)), :Array, Any, 1))::Type{Array{Any,1}},$(Expr(:call1, :(top(tuple)), :Any, :Int))::(Type{Any},Type{Int64}),Array{Any,1},0,0,0)::Array{Any,1}
        #s1924 = #s1939::Array{Any,1}
        return __sort#77__(#s1924::Array{Any,1},v::Array{Float64,1})::Array{Float64,1}
    end::Array{Float64,1}))))

b) Introspective


Julia's introspection is awesome, particularly if you enjoy looking at native assembler code. Dissecting assembler code comes in handy when optimizing algorithms. Julia programmers have several introspection functions for optimization. Here the code_native method shows the recursive nature of a binary sort algorithm.
julia> code_native(sort,(Array{Int,1},))
        .text
        push    RBP
        mov     RBP, RSP
        push    R14
        push    RBX
        sub     RSP, 48
        mov     QWORD PTR [RBP - 56], 6
        movabs  R14, 139889005508848
        mov     RAX, QWORD PTR [R14]
        mov     QWORD PTR [RBP - 48], RAX
        lea     RAX, QWORD PTR [RBP - 56]
        mov     QWORD PTR [R14], RAX
        xorps   XMM0, XMM0
        movups  XMMWORD PTR [RBP - 40], XMM0
        mov     QWORD PTR [RBP - 24], 0
        mov     RBX, QWORD PTR [RSI]
        movabs  RAX, 139888990457040
        mov     QWORD PTR [RBP - 32], 28524096
        mov     EDI, 28524096
        xor     ESI, ESI
        call    RAX
        lea     RSI, QWORD PTR [RBP - 32]
        movabs  RCX, 139889006084144
        mov     QWORD PTR [RBP - 40], RAX
        mov     QWORD PTR [RBP - 32], RAX
        mov     QWORD PTR [RBP - 24], RBX
        mov     EDI, 128390064
        mov     EDX, 2
        call    RCX
        mov     RCX, QWORD PTR [RBP - 48]
        mov     QWORD PTR [R14], RCX
        add     RSP, 48
        pop     RBX
        pop     R14
        pop     RBP
        ret

c) Multiple Dispatch

Multiple dispatch allows Object Oriented behavior.  Each function can have several  methods designed to operate on the types of the method parameters. The appropriate method is dispatched at runtime based on the parameter types.

julia> methods(sort)
# 4 methods for generic function "sort":
sort(r::UnitRange{T<:real at="" bstractarray="" dim::integer="" pre="" r::range="" range.jl:533="" range.jl:536="" sort.jl:358="" sort.jl:368="" sort="" v::abstractarray="">

Monday, August 11, 2014

Running Speedment ACE on Amazon EC2


Speedment ACE is a Graph Database Converter, and a powerful software development tool.  A Graph Database  is a NoSql database that performs best when the relationships between nodes of your data are the most important (and frequently accessed) part of that data.



Speedment ACE builds a Graph Data Grid (GDG) automatically, either from existing tables, or using the Ace Front end to generate those tables.  A great way to check out Speedment ACE is to install it on Amazon Web Services.  Here's how to get going with Speedment in 10 easy steps.

1)  Create an AWS MySql Instance.  

Note the database endpoint, name, username and password then connect form a MySql client.

2)  From the Speedment Programmer's Guide

Create a test schema and sample users table:

CREATE SCHEMA speedment_test; 
USE speedment_test; 
CREATE TABLE `speedment_test`.`user` ( 
 `id` INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
 `name` VARCHAR(45) NOT NULL,
 `surname` VARCHAR(45) NOT NULL,
 `email` VARCHAR(45) NOT NULL,
 `password` VARCHAR(45) NOT NULL,
 PRIMARY KEY (`id`), UNIQUE INDEX `Index_email`(`email`),
 INDEX `Index_name`(`name`)
) ENGINE = InnoDB;

3) Create an AWS Windows Instance

Be sure to associate the instance with a key pair so you can connect using a Windows RDP Client.  Then in the AWS Console, Launch your new instance, and generate a password using your private key. Also make not of the IP address of your new instance.

4) Connect to your new Windows Instance using an RDP client.

If you have Google Chrome, you can use the 2X Client for RDP/Remote Desktop.

5) Internet Explorer on your new EC2 Windows Instance will ask you

to add every URL you visit for your permission.  You may want to install another browser  to avoid this.

6) Download and install the JSDK 1.7.  

After installing set the JAVA_HOME environment variable to the bin directory where the 'java' executable is located.

7) Download and unzip the Speedment Ace Front End .  

Run it from the ace.bat file located in the bin directory.



8) After registering, Create a new Project.


9) Right-click on the new project and select “Add DBMS”.  

Add the info from the DB instance we created in step 1.

10) Starting on page 28 of the  Speedment Programmer's Guide

you can now explore capabilities of the Speedment Ace front end.  This includes the ability to generate code for rapid application development, and building a Graph Data Grid (GDG), the core of Speedment's optimization capabilities.







Monday, May 5, 2014

An IT Department at your fingertips : Amazon's EC2

Amazon Web Services is an essential tool for a versatile software consultant. It gives us a vast array of possible configurations. This comes in handy when working out new ideas. But it's also great for installing and testing legacy code.

EC2 - Virtual Servers in the Cloud




The EC2 tab lets us create virtual machines of many flavors. You can test out AWS for a year using the free usage tier.

Don't have a copy of windows handy? Spin up an EC2 instance with Windows, then connect using remote desktop software.  You can also use a Linux box any time you desire.

There's much that can be done with these instances, even install X-Windows on a linux instance and connect from a Chromebook using Chrome Remote Desktop.


Thursday, January 2, 2014

Running OpenTSDB on Amazon EC2

Although there are cheaper alternatives for production systems, It's easy enough to get The Open Time Series Database OpenTSDB running on an EC2 instance of Amazon Web Services.

  1. First you'll need to run HBase on EC2
  2. Make a data directory mkdir hbase_data
  3. vi hbase-0.94.13/conf/ hbase-site.xml
  4. Using vi update the hbase.rootdir property value to: file:///home/ec2-user/hbase-0.94.13/hbase-\${user.name}/hbase
  5. sudo yum install git
  6. git clone git://github.com/OpenTSDB/opentsdb.git
  7. sudo yum install automake
  8. yum install gnuplot
  9. cd opentsdb
  10. ./build.sh
  11. env COMPRESSION=NONE HBASE_HOME=path/to/hbase-0.94.X ./src/create_table.sh
  12. tsdtmp=${TMPDIR-'/tmp'}/tsd
  13. mkdir -p "$tsdtmp" 
  14. ./build/tsdb tsd --port=4242 --staticroot=build/staticroot --cachedir="$tsdtmp"
  15. In AWS, click on your EC2 instance, then click "Security Groups" at the bottom left.  Click on the default group, then click the "inbound" tab.  You can now open the ec2 port 4242. 
Your ip address on port 4242 will display the web UI for your instance of OpenTSDB:









  • Popular Articles