Spatial Mapping of Education

------------------------------------------------------------------------------------------------------
      name:  <Solomon Negash>
       log:  ~\web_spmap.smcl
  log type:  smcl
 opened on:  23 Dec 2018, 18:42:16

. **********************************************
. * Solomon Negash - How to prepare a visual Spatial Map using STATA
. * STATA Program, version 15.1. 
. * Dec 21, 2018
. ******************** SETUP *********************

. *Data source Central Statistics Agency of Ethiopia
. *The current work assumes all necessarily files are available in the current directory (cd)
 
. import  delimited edu.csv 

. save "edu.dta"
file edu.dta saved

. *NB. Education data for Year 2000 is not available. For convinience we take average values*/

. // Step 1b. Convert wide into long format and save  
. destring private2002, replace 
private2002: all characters numeric; replaced as long
(3 missing values generated)
. destring public2002, replace 
public2002: all characters numeric; replaced as long
(2 missing values generated)

. reshape long private public, i(id) j(year)
(note: j = 1994 1995 1996 1997 1998 1999 2000 2001 2002)

Data                               wide   ->   long
-----------------------------------------------------------------------------
Number of obs.                       11   ->      99
Number of variables                  21   ->       6
j variable (9 values)                     ->   year
xij variables:
private1994 private1995 ... private2002   ->   private
   public1994 public1995 ... public2002   ->   public
-----------------------------------------------------------------------------

. save "edulong.dta"
file edulong.dta saved


. //Step 2. Download shapefile (country = Ethiopia) 
. *Source: https://gadm.org/country 
> *The country-level-administrative-boundaries-map comes with three admin levels:Admin1 Admin2 & Admin3
> *For our purpose we use Admin1 (which is the same as "regions" of Ethiopia) (file name "ETH_adm1")


. //Step 3. Install spmap & shp2dta 

. * ssc install spmap
. * ssc install shp2dta

. //Step 4. Convert Shape file to dta (stata) format using shp2dta command

. shp2dta using ETH_adm1, database(ethdata) coordinates(ethcoord) genid(id)

. /* The command creates a database and a coordinate file and  
> saves them as "ethdata" and "ethcoord", respectivelly. It also generates a common id to both files. 
> This id will be used later to mapping our economic data to the coordinates.*/

. use ethdata,clear

. gen regcode="AA"

. replace regcode = "AA" if NAME_1 =="Addis Ababa"
(0 real changes made)
. replace regcode = "AF" if NAME_1 =="Afar"
(1 real change made)
. replace regcode = "AM" if NAME_1 =="Amhara"
(1 real change made)
. replace regcode = "BG" if NAME_1 =="Benshangul-Gumaz"
(1 real change made)
. replace regcode = "DD" if NAME_1 =="Dire Dawa"
(1 real change made)
. replace regcode = "GM" if NAME_1 == "Gambela Peoples"
(1 real change made)
. replace regcode = "HR" if NAME_1 =="Harari People"
(1 real change made)
. replace regcode = "OR" if NAME_1 =="Oromia"
(1 real change made)
. replace regcode = "SM" if NAME_1 =="Somali"
(1 real change made)
. replace regcode = "SNNP" if NAME_1 =="Southern Nations, Nationalities and Peoples"
(1 real change made)
. replace regcode = "TG" if NAME_1 =="Tigray"
(1 real change made)
. save, replace 
file ethdata.dta saved

 




. //Step 5a 
. *Merge the converted data (ethdata.dta) with the statistical data (edulong.dta). 
. *The common variable in this case is regcode (regional Code: TG AF AM OR SM BG SNNP GM HR AA DD)
.  
. merge 1:m regcode using "edulong.dta" 

    Result                           # of obs.
    -----------------------------------------
    not matched                             0
    matched                                99  (_merge==3)
    -----------------------------------------

. save "edulong_ethdata.dta"
file edulong_ethdata.dta saved


. //Step 5b. Premapping Preparations :
.  
. //Convert data to scales of ten-thousands 
. gen spublic=public/10000
(2 missing values generated)
. label var spublic "Students in Public Schools"

. gen sprivate=private/10000
(3 missing values generated)
. label var sprivate "Students in Private Schools"

. gen sedut=(public+private)/10000
(3 missing values generated)
. label var sedut "Students in Public and Private Schools"
. save, replace
file edulong_ethdata.dta saved

. //Testing single maping = Total number of students in school by region 
. spmap sedu using ethcoord if year==1997, id(id) fcolor(Blues) 
. graph export test1.png, replace 



. //Step 6. Generate Graphs using loop function 
. /* The loop funtion below generates 9 timeseries maps for each of the three variables raning from 1994 - 2002.
> The output maps will be saved in the current directory.*/

. use edulong_ethdata, clear

. foreach x of var spublic sprivate sedut {
  2. local z: variable label `x'
  3. forvalues year = 1994(1)2002{
  4.    spmap `x' using ethcoord.dta if year==`year', id(id) fcolor(Greens2) clmethod(unique) ndfcolor(gray) title("`z'" `year', color(blue)) ///
>    ysize(8) xsize(14) legt("Legend") legend(pos(1)  size(*.95) symx(*.95) symy(*.95) forcesize ) leg o(hilo)  
>note(Source CSA 2018. www.solomonegash.com, pos(7) size(*.95) color(blue) )
  5.     local year=string(`year'-1994, "%03.0f")
  6.         graph export `x'_`year'.png, replace    
  7. }
  8. }
(note: file spublic_001.png not found)
(file spublic_001.png written in PNG format)
(...)
(note: file sedut_009.png not found)
(file sedut_009.png written in PNG format)
Example:


. //Step 7. Labeling the Maps

. //7A. Add geographic cordinates of the capitals of each region

. gen y=.                                                         
(99 missing values generated)

.                                                                 
. replace y       =       8.9806          if      regcode ==      "AA"
(9 real changes made)
. replace y       =       11.7559         if      regcode ==      "AF"
(9 real changes made)
. replace y       =       11.5742         if      regcode ==      "AM"
(9 real changes made)
. replace y       =       10.7803         if      regcode ==      "BG"
(9 real changes made)
. replace y       =       9.8009          if      regcode ==      "DD"
(9 real changes made)
. replace y       =       7.9220          if      regcode ==      "GM"
(9 real changes made)
. replace y       =       9.2126          if      regcode ==      "HR"
(9 real changes made)
. replace y       =       9.0893          if      regcode ==      "OR"
(9 real changes made)
. replace y       =       6.701835        if      regcode ==      "SM"
(9 real changes made)
. replace y       =       6.925621        if      regcode ==      "SNNP"
(9 real changes made)
. replace y       =       13.4936         if      regcode ==      "TG"
(9 real changes made)
. rename  y       ycoord                                          

. gen x=.                                                         
(99 missing values generated)
. replace x       =       38.7578         if      regcode ==      "AA"
(9 real changes made)
. replace x       =       40.9587         if      regcode ==      "AF"
(9 real changes made)
. replace x       =       37.3614         if      regcode ==      "AM"
(9 real changes made)
. replace x       =       35.5658         if      regcode ==      "BG"
(9 real changes made)
. replace x       =       41.8501         if      regcode ==      "DD"
(9 real changes made)
. replace x       =       34.1532         if      regcode ==      "GM"
(9 real changes made)
. replace x       =       43.0227         if      regcode ==      "HR"
(9 real changes made)
. replace x       =       36.5554         if      regcode ==      "OR"
(9 real changes made)
. replace x       =       44.167892       if      regcode ==      "SM"
(9 real changes made)
. replace x       =       37.148546       if      regcode ==      "SNNP"
(9 real changes made)
. replace x       =       39.4657         if      regcode ==      "TG"
(9 real changes made)
. rename  x       xcoord                                          
. save, replace
file edulong_ethdata.dta saved


. //7B. Generate Labels

. qui foreach i of var spublic sprivate sedut {
  2. use "edulong_ethdata.dta", clear
  3. g labtype_`i' =2
  4. forvalues year = 1994(1)2002 {
  5. replace labtype =1 if year==`year' 
  6. }
  7. append using "edulong_ethdata.dta"
  8. replace labtype = 2 if labtype==.
  9. replace region = string(`i', "%4.1f") if labtype == 2
 10. keep xcoor ycoor region labtype year
 11. rename region reg
 12. sort year reg
 13. save spmaplabels_`i', replace
 14. }
(...)
file spmaplabels_sedut.dta saved

 




. //8C. Generate the timeseries maps with labels

. use "edulong_ethdata.dta", clear

. foreach x of var spublic sprivate sedut {
  2. local z: variable label `x'
  3. forvalues year = 1994(1)2002{
  4.    spmap `x' using "ethcoord.dta" if year==`year', id(id) fcolor(Rainbow) ocolor(gs10 gs9 gs8 gs7
> ) clmethod(unique)  label(data(spmaplabels_`x') xcoord(xcoord)  ycoord(ycoord) label(reg) by(labtype
> ) size(0.95 ..) pos(12 0)) title("`z'" `year', color(blue)) ysize(8) xsize(14) legt("Legend") legend
> (pos(1)  size(*.95) symx(.95) symy(.95) forcesize ) lego(hilo)  note(CSA 2018. www.solomonegash.com,
>  pos(7) size(*.85) color(blue)) name(`x'_`year'l)
  5.     local year=string(`year'-1993, "%03.0f")
  6.         graph export `x'_`year'l.png, replace   
  7. }
  8. }
(note: file spublic_001l.png not found)
(file spublic_001l.png written in PNG format)
(...27 graphs)
(note: file sedut_009l.png not found)
(file sedut_009l.png written in PNG format)
Example:


. forvalues year = 1994(1)2002{
  2. graph combine spublic_`year'l sprivate_`year'l, cols(2) name(comp`year', replace)
  3. graph export comp`year'l.png, replace
  4. }
(note: file comp1994l.png not found)
(file comp1994l.png written in PNG format)
(...9 graphs ...)
(note: file comp2002l.png not found)
(file comp2002l.png written in PNG format)
Example:


. //drop all graph in memory
. graph drop _all


//8. Convert graphs to video or gif for better visualisation 

*Step 1 - Download FFMPEG from https://www.ffmpeg.org/
*Step 2 - Extract it to desktop and 
*Step 3 - Locate the application ffmpeg.ex. 
*Step 4  -  But first locate graphpath (current directory) 

//On Windows - we can excute the following command directly from STATA command
local Gpath "C:\Users\mywin\Desktop\ethedu_web"
winexec "C:\Users\mywin\Desktop\ffmpeg\bin\ffmpeg.exe" -i `Gpath'sedut_%03d.png -b:v 512k `Gpath'graph.mpg




winexec "C:\Users\mywin\Desktop\ffmpeg\bin\ffmpeg.exe" -r 10 -i `GraphPath'graph.mpg -t 5 -r 5 `Gpath'graph.gif 


 




 //On Mac - Execute it from Terminal 
*Convert the graphs into mp4, mpg, etc
~/Desktop/ffmpeg/ffmpeg -r 2 -i ~/Desktop/ethedu_web/sedut_%03d.png -pix_fmt yuv420p sedut.mp4
~/Desktop/ffmpeg/ffmpeg -r 5 -i /Desktop/ethedu_web/sedut.mp4 -t 5 -r 5 redut.gif

. log close 
      name:  <Solomon Negash>
       log:  ~\web_spmap.smcl
  log type:  smcl
 closed on:  23 Dec 2018, 18:44:06
------------------------------------------------------------------------------------------------------