Follow by Email

Translate

Thursday, March 19, 2015

Introduction to Julia

"Julia is a fresh approach to technical computing."  boasts the startup message, flourished with colorful circles hovering above a bubbly ASCII Julia logo.  The formatting effort is not wasted, it's an exuberant promise: Julia will make the command line fun again.
apptrain_1@julia:~/workspace $ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.1 (2014-02-11 06:30 UTC)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |  x86_64-linux-gnu

julia> 

Julia was created by four Data Scientists from MIT who began working on it around 2011.  The language is beginning to mature at a time when the Data Scientist job title is popping up on resumes as fast as Data Scientist jobs appear.  The timing is excellent.   R programming, an offshoot of S Programming , is the language of choice for today's mathematical programmer.  But it feels clunky, like a car from the last century. While Julia may not unseat R in the world of Data Analysis,  plans don't stop there.

If you want to code along with the examples in this article, jump to Getting Started with Julia and chose one of the three options to start coding.

Julia is a general purpose programming language.  It's creators have noble goals.  They want a language that is fast like C, they want it flexible with cool metaprograming capabilities like Ruby, they want parallel and distributed computing like Scala, and true Mathematical equations like MATLAB.



Why program in Julia?

1) Julia is Fast


Julia already boasts faster matrix multiplication and sorting than Go and Java.  It uses the LLVM compiler, which languages like GO use for fast compilation.   Julia uses just in time (JIT) compilation to machine code , and often achieves C like performance numbers.

2) Julia is written in Julia

Contributors need only work with a single language, which makes it easier for Julia users to become core contributors. 
"As a policy, we try to never resort to implementing things in C. This keeps us honest – we have to make Julia fast enough to allow us to do that" -Stephan Karpinski

And, as the languages co-creator Karpinski notes in the comments of the referenced post,   Writing the language itself in Julia means that when improvements are made to the compiler, both the system and user code gets faster.

3) Julia is Powerful

Like most programming languages, it's implementation is Open Source.  Anyone can work on the language or the documentation.  And like most modern programming languages, Julia has extensive metaprogramming support.  It's creators attribute the Lisp language for their inspiration:
Like Lisp, Julia represents its own code as a data structure of the language itself.
a) Optional Strong Typing
Using strong typing can speed up compiling, but Julia keeps strong typing optional, which frees up programmers who want to write dynamic routines that work on multiple types. 
julia> @code_typed(sort(arry))
1-element Array{Any,1}:
 :($(Expr(:lambda, {:v}, {{symbol("#s1939"),symbol("#s1924")},{{:v,Array{Float64,1},0},{symbol("#s1939"),Array{Any,1},18},{symbol("#s1924"),Array{Any,1},18}},{}}, :(begin $(Expr(:line, 358, symbol("sort.jl"), symbol("")))
        #s1939 = (top(ccall))(:jl_alloc_array_1d,$(Expr(:call1, :(top(apply_type)), :Array, Any, 1))::Type{Array{Any,1}},$(Expr(:call1, :(top(tuple)), :Any, :Int))::(Type{Any},Type{Int64}),Array{Any,1},0,0,0)::Array{Any,1}
        #s1924 = #s1939::Array{Any,1}
        return __sort#77__(#s1924::Array{Any,1},v::Array{Float64,1})::Array{Float64,1}
    end::Array{Float64,1}))))

b) Introspective


Julia's introspection is awesome, particularly if you enjoy looking at native assembler code. Dissecting assembler code comes in handy when optimizing algorithms. Julia programmers have several introspection functions for optimization. Here the code_native method shows the recursive nature of a binary sort algorithm.
julia> code_native(sort,(Array{Int,1},))
        .text
        push    RBP
        mov     RBP, RSP
        push    R14
        push    RBX
        sub     RSP, 48
        mov     QWORD PTR [RBP - 56], 6
        movabs  R14, 139889005508848
        mov     RAX, QWORD PTR [R14]
        mov     QWORD PTR [RBP - 48], RAX
        lea     RAX, QWORD PTR [RBP - 56]
        mov     QWORD PTR [R14], RAX
        xorps   XMM0, XMM0
        movups  XMMWORD PTR [RBP - 40], XMM0
        mov     QWORD PTR [RBP - 24], 0
        mov     RBX, QWORD PTR [RSI]
        movabs  RAX, 139888990457040
        mov     QWORD PTR [RBP - 32], 28524096
        mov     EDI, 28524096
        xor     ESI, ESI
        call    RAX
        lea     RSI, QWORD PTR [RBP - 32]
        movabs  RCX, 139889006084144
        mov     QWORD PTR [RBP - 40], RAX
        mov     QWORD PTR [RBP - 32], RAX
        mov     QWORD PTR [RBP - 24], RBX
        mov     EDI, 128390064
        mov     EDX, 2
        call    RCX
        mov     RCX, QWORD PTR [RBP - 48]
        mov     QWORD PTR [R14], RCX
        add     RSP, 48
        pop     RBX
        pop     R14
        pop     RBP
        ret

c) Multiple Dispatch

Multiple dispatch allows Object Oriented behavior.  Each function can have several  methods designed to operate on the types of the method parameters. The appropriate method is dispatched at runtime based on the parameter types.

julia> methods(sort)
# 4 methods for generic function "sort":
sort(r::UnitRange{T<:real at="" bstractarray="" dim::integer="" pre="" r::range="" range.jl:533="" range.jl:536="" sort.jl:358="" sort.jl:368="" sort="" v::abstractarray="">

Thursday, February 12, 2015

JULIA – A FAST DYNAMIC LANGUAGE FOR TECHNICAL COMPUTING by ├średev Conference

Julia is a high-level, high-performance dynamic language for scientific computing. It has been gaining traction as a faster alternative to Matlab, R and NumPy and as a more productive alternative to C, C++ and Fortran. Julia is particularly relevant when both expressiveness and performance are paramount – in areas like machine learning, “big statistics”, linear algebra, bioinformatics, and image analysis.

Monday, February 9, 2015

NFL Fantasy Sports API

The NFL has made an impressive Application Programming Interface (API) available to application developers.



 Fantasty Football Web Services  allows existing or new applications to access live NFL data, potentially in real time.  It also allows users to join existing fantasy leagues, or even create new leagues.  Perhaps the most exciting offerings are still to come. Michael Vizard  of Programmable Web writes:

A big part of that effort revolves around giving fans access to statistics and analytics tools that they can use to figure out which players to draft and keep. In the postseason, NFL.com even went so far as to create a separate fantasy game event that involved just the teams that made the playoffs.

Access to these statistics and analytic tools is something large organizations like the NFL need to compete with the powerful data analysis capabilities available to smaller companies today. In addition to the  playoff fantasy game mentioned above, there's also a prowbowl api available. So the NFL seems to be running with the API.

First, let's have a closer look at the The Web services available now which are documented here: http://api.fantasy.nfl.com/v1/docs .  Then we'll explore some hidden gems available now in the API that access some underlying predictive analytics, straight form the NFL!

Available Data

Any current player statistics are available through the well documented API calls. To write to the API you'll need a key which you request by emailing fantasy.football@nfl.com .




Scoring Leaders


{"season":"2010","week":"","positions":{"QB":[{"id":"2505996","esbid":"MAN473170","gsisPlayerId":"00-0022803","firstName":"Eli","lastName":"Manning","teamAbbr":"NYG","opponentTeamAbbr":"","position":"QB","rank":"11","statsLine":"","stats":{"PassYds":false,"PassTDs":false,"Int":false,"RushYds":false,"RushTDs":false,"RecYds":false,"RecTds":false,"FumTD":false,"TwoPt":false,"FumLost":false},"pts":"231.08","projectedPts":false,"status":""},{"id":"2539961","esbid":"NAS738294","gsisPlayerId":"00-0030586","firstName":"Ryan","lastName":"Nassib","teamAbbr":"NYG","opponentTeamAbbr":"","position":"QB","rank":"47","statsLine":"","stats":{"PassYds":false,"PassTDs":false,"Int":false,"RushYds":false,"RushTDs":false,"RecYds":false,"RecTds":false,"FumTD":false,"TwoPt":false,"FumLost":false},"pts":false,"projectedPts":false,"status":""}]}}

The data comes back in XML by default, but you can clean that up with a simple format parameter at the end of your request:




Weekly Stats

Users can further filter requests by position, team, week and season.  The available  stats go back to 2009.




Advanced Stats

Additional statistics such as RedZone Touches are also available




{"QB":[{"id":"2533033","esbid":"GRI283140","gsisPlayerId":"00-0029665","firstName":"Robert","lastName":"Griffin","teamAbbr":"WAS","opponentTeamAbbr":"@HOU","position":"QB","stats":{"FanPtsAgainstOpponentPts":"25.00","FanPtsAgainstOpponentRank":"2","Carries":"9","Touches":"9","Receptions":false,"Targets":false,"ReceptionPercentage":false,"RedzoneTargets":false,"RedzoneTouches":"1","RedzoneG2g":false},"status":"Loss, 6-17"}],"RB":[{"id":"2533457","esbid":"MOR317547","gsisPlayerId":"00-0029141","firstName":"Alfred","lastName":"Morris","teamAbbr":"WAS","opponentTeamAbbr":"@HOU","position":"RB","stats":{"FanPtsAgainstOpponentPts":"25.60","FanPtsAgainstOpponentRank":"4","Carries":"28","Touches":"28","Receptions":false,"Targets":false,"ReceptionPercentage":false,"RedzoneTargets":false,"RedzoneTouches":"4","RedzoneG2g":"2"},"status":"Loss, 6-17"}],"WR":[{"id":"80425","esbid":"HAR829482","gsisPlayerId":"00-0026998","firstName":"Percy","lastName":"Harvin","teamAbbr":"NYJ","opponentTeamAbbr":"OAK","position":"WR","stats":{"FanPtsAgainstOpponentPts":"39.00","FanPtsAgainstOpponentRank":"3","Carries":"5","Touches":"11","Receptions":"6","Targets":"8","ReceptionPercentage":"75","RedzoneTargets":"1","RedzoneTouches":"2","RedzoneG2g":"1"},"status":"Win, 19-14"}],"TE":[{"id":"2530473","esbid":"ADA482150","gsisPlayerId":"00-0028337","firstName":"Kyle","lastName":"Adams","teamAbbr":"","opponentTeamAbbr":"Bye","position":"TE","stats":{"FanPtsAgainstOpponentPts":"","FanPtsAgainstOpponentRank":"","Carries":false,"Touches":"1","Receptions":"1","Targets":"1","ReceptionPercentage":"100","RedzoneTargets":false,"RedzoneTouches":false,"RedzoneG2g":false},"status":""}],"K":[{"id":"2499370","esbid":"AKE551610","gsisPlayerId":"00-0000108","firstName":"David","lastName":"Akers","teamAbbr":"","opponentTeamAbbr":"Bye","position":"K","stats":{"FanPtsAgainstOpponentPts":"","FanPtsAgainstOpponentRank":"","Carries":false,"Touches":false,"Receptions":false,"Targets":false,"ReceptionPercentage":false,"RedzoneTargets":false,"RedzoneTouches":false,"RedzoneG2g":false},"status":""}],"DEF":[{"id":"100029","esbid":false,"gsisPlayerId":false,"firstName":"San Francisco","lastName":"49ers","teamAbbr":"SF","opponentTeamAbbr":"@DAL","position":"DEF","stats":{"FanPtsAgainstOpponentPts":"5.00","FanPtsAgainstOpponentRank":"20","Carries":false,"Touches":false,"Receptions":false,"Targets":false,"ReceptionPercentage":false,"RedzoneTargets":false,"RedzoneTouches":false,"RedzoneG2g":false},"status":"Win, 28-17"}]}


Managing Leagues with API Writes


Add a Player

This call adds a player to your fantasy team.


A valid API key will get a success response:

    success
    true


Create a League

You can even create a new league,


then email out links for people to join:




For the application developer and fantasy sports fan, the fun has just begun.

Analytic Tools


Developers and analysts are used to writing their own analytic tools.  The API provides data designed just for custom analytics.  The Pro Bowl API returns players twitter user ids. Those feeds along with the players/news call can keep users up to date with the latest developments.  Potentially that text data can even be mined and analyzed for predictors of next weeks performance.  Does Gronk play better after appearing on Conan?







Predictions

Predictive analytics has become so prevalent that the NFL is now providing projections of next weeks fantasy points.

weekProjectedPts

Have a look at the JSON response to this request for 2014 week stats.


Algorithms behind the scenes at the NFL boldly predicted at week 1 that NY Giants backup Quarterback Husain Abdullah would have no points the next week (weekProjectedPts)


{"id":"729","esbid":"ABD660476","gsisPlayerId":"00-0025940","name":"Husain Abdullah","position":"DB","teamAbbr":"KC","stats":{"1":"16","70":"58","71":"13","73":"1","76":"1","81":"10","82":"39","84":"2","85":"5","89":"1"},"seasonPts":82.5,"seasonProjectedPts":0,"weekPts":3,"weekProjectedPts":0}




The NFL algorithms were close. If you change the week number to 2 in the url above, you'll see that Husain did scrape up a point the next week.


seasonProjectedPts

Season projected points appears in the /playesrs/stats response as well. It appears to be a placeholder for now.  It doesn't change week to week, and it's 0 for any year prior to 2014.  It will be interesting to watch this attribute in 2015.

Watch for more updates to the NFL Fantasy Sports API during the offseason, and for some interesting applications build around the new API.


Popular Articles