DOL-ILAB SDC - Section_0
Pablo Diego-Rosell, PhD
2021-08-25
- Setup filenames
- Setup data, functions and create dictionary for dataset review
- Direct PII: variables to be removed
- Direct PII-team: Encode field team names
- Small locations: Encode locations with pop <100,000 using random large numbers
- Indirect PII - Ordinal: Global recode or Top/bottom coding for extreme values
- Indirect PII - Categorical: Recode, encode, or Top/bottom coding for extreme values
- Matching and crosstabulations: Run automated PII check
- Open-ends: review responses for any sensitive information, redact as necessary
- GPS data: Displace
- Save processed data in Stata and SPSS format
rm(list=ls(all=t))Setup filenames
filename <- "Section_0" # !!!Update filename
functions_vers <- "functions_1.8.R" # !!!Update helper functions fileSetup data, functions and create dictionary for dataset review
source (functions_vers)## --------
## This is sdcMicro v5.6.0.
## For references, please have a look at citation('sdcMicro')
## Note: since version 5.0.0, the graphical user-interface is a shiny-app that can be started with sdcApp().
## Please submit suggestions and bugs at: https://github.com/sdcTools/sdcMicro/issues
## --------##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union## Loading required package: sp## Checking rgeos availability: TRUE##
## Attaching package: 'raster'## The following object is masked from 'package:dplyr':
##
## select## The following object is masked from 'package:sdcMicro':
##
## freq## rgdal: version: 1.5-23, (SVN revision 1121)
## Geospatial Data Abstraction Library extensions to R successfully loaded
## Loaded GDAL runtime: GDAL 3.2.1, released 2020/12/29
## Path to GDAL shared files: C:/Users/C_Pablo_Diego-Rosell/Documents/R/R-3.6.3/library/rgdal/gdal
## GDAL binary built with GEOS: TRUE
## Loaded PROJ runtime: Rel. 7.2.1, January 1st, 2021, [PJ_VERSION: 721]
## Path to PROJ shared files: C:/Users/C_Pablo_Diego-Rosell/Documents/R/R-3.6.3/library/rgdal/proj
## PROJ CDN enabled: FALSE
## Linking to sp version:1.4-5
## To mute warnings of possible GDAL/OSR exportToProj4() degradation,
## use options("rgdal_show_exportToProj4_warnings"="none") before loading rgdal.
## Overwritten PROJ_LIB was C:/Users/C_Pablo_Diego-Rosell/Documents/R/R-3.6.3/library/rgdal/proj## Loading required package: spatstat.data## Loading required package: spatstat.geom## spatstat.geom 2.0-1##
## Attaching package: 'spatstat.geom'## The following objects are masked from 'package:raster':
##
## area, rotate, shift## Loading required package: spatstat.core## Loading required package: nlme##
## Attaching package: 'nlme'## The following object is masked from 'package:raster':
##
## getData## The following object is masked from 'package:dplyr':
##
## collapse## Loading required package: rpart## spatstat.core 2.0-0## Loading required package: spatstat.linnet## spatstat.linnet 2.1-1##
## spatstat 2.0-1 (nickname: 'Caution: contains small parts')
## For an introduction to spatstat, type 'beginner'## rgeos version: 0.5-5, (SVN revision 640)
## GEOS runtime version: 3.8.0-CAPI-1.13.1
## Linking to sp version: 1.4-4
## Polygon checking: TRUE##
## Spatial Point Pattern Analysis Code in S-Plus
##
## Version 2 - Spatial and Space-Time analysis##
## Attaching package: 'splancs'## The following object is masked from 'package:raster':
##
## zoom## The following object is masked from 'package:dplyr':
##
## tribble## Loading required package: spam## Loading required package: dotCall64## Loading required package: grid## Spam version 2.6-0 (2020-12-14) is loaded.
## Type 'help( Spam)' or 'demo( spam)' for a short introduction
## and overview of this package.
## Help for individual functions is also obtained by adding the
## suffix '.spam' to the function name, e.g. 'help( chol.spam)'.##
## Attaching package: 'spam'## The following objects are masked from 'package:base':
##
## backsolve, forwardsolve## See https://github.com/NCAR/Fields for
## an extensive vignette, other supplements and source code##
## Attaching package: 'geosphere'## The following object is masked from 'package:spatstat.geom':
##
## perimeter##
## Attaching package: 'tibble'## The following object is masked from 'package:splancs':
##
## tribbleVisually inspect variables in "dictionary.csv" and flag for risk, using the following flags:
# Direct PII: Respondent Names, Addresses, Identification Numbers, Phone Numbers
# Direct PII-team: Interviewer Names, other field team names
# Indirect PII-ordinal: Date of birth, Age, income, education, household composition.
# Indirect PII-categorical: Gender, education, ethnicity, nationality,
# occupation, employer, head of household, marital status
# GPS: Longitude, Latitude
# Small Location: Location (<100,000)
# Large Location (>100,000)
# Weight: weightVar
# Household ID: hhId,
# Open-ends: Review responses for any sensitive information, redact as necessary Direct PII: variables to be removed
# !!!No Direct PIIDirect PII-team: Encode field team names
# !!!Replace vector in "variables" field below with relevant variable names
mydata <- encode_direct_PII_team (variables=c("enumerator"))## [1] "Frequency table before encoding"
## enumerator. Field Officer: What is your name?
## 1. Joel Batalla 2. John Mark Marquez 3. Darwin Ceruma
## 91 73 163
## 4. Raymond Kenneth G. Agner 5. Kevin Clidoro 6. Norshid Ukat Talib
## 37 92 83
## 10. Joseph Almel Escaro Bausa 12. Benella Madarang 13. Angelica Caponpon
## 39 56 45
## 14. Rossan Ignacio 15. Jillian Hurtada 16. Almira Joy Sumampong
## 50 17 16
## 17. Annalyn Verzosa 18. Shirley Mendros 19. Emily Aspe
## 62 57 82
## 20. Verna Patricia Fuentebella 21. Bernadette Suello Almanzor 22. Maryjane Samonte
## 4 3 92
## 23. Nor-En Lambac 24. Stephanie Canopin Catalogo 25. Lorna A. Mabilangan
## 2 56 81
## 26. Aisa Punay 27. Sylvia Alinsunurin Cruz 28. Jessrel Alba
## 76 26 15
## 29. Ma. Nadia S. Villanueva 30. Joebelle A. Gasang 31. Allanie Bedruz Alcantara
## 93 68 95
## 32. Ronalyn G. Valera 33. Maria Melanie V. Bagwang 34. Ma.Riza Eudela Caldeo
## 70 73 61
## 35. Hazel Deen A. Magno 36. Joereen Belardo Nesas 37. Marian Tarrayo
## 82 77 4
## 41. Lady Lou V. Dawaten 42. Other 43. Farah Lacerna
## 87 18 92
## 44. ANGELENE KIM LABATETE LLANTO 45. Glaiza Marie Retoya Principe 46. HONEYLEEN ABADA LOILO
## 19 20 28
## 47. Mary Grace Lucas Mata 48. MARY JEAN FUENTES UMLAS 49. Mary Lyn Lopez
## 7 36 28
## 50. Michell Escasinas Macali
## 20
## [1] "Frequency table after encoding"
## enumerator. Field Officer: What is your name?
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
## 91 73 163 37 92 83 39 56 45 50 17 16 62 57 82 4 3 92 2 56 81 76 26 15 93 68 95 70 73
## 30 31 32 33 34 35 36 37 38 39 40 41 42 43
## 61 82 77 4 87 18 92 19 20 28 7 36 28 20Small locations: Encode locations with pop <100,000 using random large numbers
# !!!Include relevant variables, but check their population size first to confirm they are <100,000
locvars <- c("municipality",
"barangay_id")
mydata <- encode_location (variables= locvars, missing=999999)## [1] "Frequency table before encoding"
## municipality. Municipality
## Abucay Agno Amulung Anda
## 126 56 70 42
## Bani Bautista Bugallon Calabanga
## 28 14 56 70
## Calasiao Camaligan Canaman Candelaria
## 14 14 42 28
## Cauayan City Enrile General Emilio Aguinaldo Jala-Jala
## 56 70 42 42
## Jones Jose Panganiban Labo Libmanan
## 252 70 56 14
## Magarao Malinao Manito Mariveles
## 42 168 42 224
## Naga City Ocampo Pagsanjan Pasacao
## 28 14 126 42
## Pila Pilar Pililla Polangui
## 14 70 42 56
## Sampaloc San Carlos City San Mateo San Nicolas
## 56 14 42 28
## Sorsogon City Sual Tanay Tinambac
## 28 14 56 14
## Urbiztondo
## 14
## [1] "Frequency table after encoding"
## municipality. Municipality
## 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816
## 42 70 42 28 56 28 70 42 14 42 70 14 28 126 42 56 14 252 14 14 56 56 56 28 126 70 14 42 42
## 817 818 819 820 821 822 823 824 825 826 827 828
## 42 42 56 14 224 70 168 14 14 14 28 56
## [1] "Frequency table before encoding"
## barangay_id. group(barangay)
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## [1] "Frequency table after encoding"
## barangay_id. group(barangay)
## 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
## 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997
## 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14mydata <- mydata[ , !names(mydata) %in% "cen_total_pop"] # Drop as it seems like the population size of the location in the census, which would be akin to a small area identifier. Indirect PII - Ordinal: Global recode or Top/bottom coding for extreme values
# Focus on variables with a "Lowest Freq" in dictionary of 30 or less.
# Top code household composition variables with large and low and unusual numbers
mydata <- top_recode ("numinhh", break_point=12, missing=c(888, 999999)) ## [1] "Frequency table before encoding"
## numinhh. How many people live in this household, including you? Ilang tao ang naniniraha
## 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 19
## 13 89 317 437 446 390 252 160 95 42 21 17 3 4 6 2 2
## [1] "Frequency table after encoding"
## numinhh. How many people live in this household, including you? Ilang tao ang naniniraha
## 2 3 4 5 6 7 8 9 10 11
## 13 89 317 437 446 390 252 160 95 42
## 12 or more
## 55
mydata <- under_recode ("numinhh", break_point=3, missing=c(888, 999999)) ## [1] "Frequency table before encoding"
## numinhh. How many people live in this household, including you? Ilang tao ang naniniraha
## 2 3 4 5 6 7 8 9 10 11
## 13 89 317 437 446 390 252 160 95 42
## 12 or more
## 55
## [1] "Frequency table after encoding"
## numinhh. How many people live in this household, including you? Ilang tao ang naniniraha
## 3 or less 4 5 6 7 8 9 10 11 12 or more
## 102 317 437 446 390 252 160 95 42 55
Indirect PII - Categorical: Recode, encode, or Top/bottom coding for extreme values
# !!!No Indirect PII - Categorical
# Recode those with very specific values. Matching and crosstabulations: Run automated PII check
# !!!Insufficient demographic dataOpen-ends: review responses for any sensitive information, redact as necessary
# !!!No Open-ends GPS data: Displace
# !!!No GPS dataSave processed data in Stata and SPSS format
haven::write_dta(mydata, paste0(filename, "_PU.dta"))
haven::write_sav(mydata, paste0(filename, "_PU.sav"))
# Add report title dynamically
title_var <- paste0("DOL-ILAB SDC - ", filename)