Samuel Bosch

Updating Drupal Modules with the Update Manager on Windows IIS: Cannot remove file

2019-01-08T10:17:00.000+01:00

While trying to update the editor_file Drupal module with the web interface I encountered the following error:

Cannot remove file /web/modules/editor_file\config\schema\editor_file.schema.yml

After a long search I remembered that I had changed the application pool user in IIS for my website but the corresponding file permissions where apparently not updated. You can set file permissions for the application pool user b:

selecting the directory in IIS
clicking Edit Permissions
going to the tab Security
clicking Edit
clicking Add...
the object name for app pool users is the name of the application pool prefixed by IIS AppPool\ e.g. IIS AppPool\mywebsite
I gave Full Control permissions to the module folder and my update procedure worked

Some system notes:

Drupal 8.6
Windows Server 2012 R2

I'm new to Drupal so if you found a better way let me know.

Great circle calculations with numpy

2018-09-11T23:11:00.000+02:00

In this very short post I want to point you to some code for calculating the centroid and distance to that centroid for a set of points in numpy. As bonus I also included some profiling. Note that I already blogged about the centroid function in a previous post.

from math import atan2, sqrt, degrees
import numpy as np
from math import radians, sin, cos

RADIUS = 6371.009


def get_centroid(points):
    xy = np.asarray(points)
    xy = np.radians(xy)
    lon, lat = xy[:, 0], xy[:, 1]
    avg_x = np.sum(np.cos(lat) * np.cos(lon)) / xy.shape[0]
    avg_y = np.sum(np.cos(lat) * np.sin(lon)) / xy.shape[0]
    avg_z = np.sum(np.sin(lat)) / xy.shape[0]
    center_lon = atan2(avg_y, avg_x)
    hyp = sqrt(avg_x * avg_x + avg_y * avg_y)
    center_lat = atan2(avg_z, hyp)
    return degrees(center_lon), degrees(center_lat)


def gc_distance_points(a, points):
    b = np.asarray(points)
    lat1, lng1 = radians(a[1]), radians(a[0])
    lat2, lng2 = np.radians(b[:, 1]), np.radians(b[:, 0])

    sin_lat1, cos_lat1 = sin(lat1), cos(lat1)
    sin_lat2, cos_lat2 = np.sin(lat2), np.cos(lat2)

    delta_lng = np.subtract(lng2, lng1)
    cos_delta_lng, sin_delta_lng = np.cos(delta_lng), np.sin(delta_lng)

    d = np.arctan2(np.sqrt((cos_lat2 * sin_delta_lng) ** 2 +
                 (cos_lat1 * sin_lat2 -
                  sin_lat1 * cos_lat2 * cos_delta_lng) ** 2),
                 sin_lat1 * sin_lat2 + cos_lat1 * cos_lat2 * cos_delta_lng)

    return RADIUS * d


def gc_dist(a, b):
        from math import radians, sin, cos, sqrt, atan2
        lat1, lng1 = radians(a[1]), radians(a[0])
        lat2, lng2 = radians(b[1]), radians(b[0])

        sin_lat1, cos_lat1 = sin(lat1), cos(lat1)
        sin_lat2, cos_lat2 = sin(lat2), cos(lat2)

        delta_lng = lng2 - lng1
        cos_delta_lng, sin_delta_lng = cos(delta_lng), sin(delta_lng)

        d = atan2(sqrt((cos_lat2 * sin_delta_lng) ** 2 +
                       (cos_lat1 * sin_lat2 -
                        sin_lat1 * cos_lat2 * cos_delta_lng) ** 2),
                  sin_lat1 * sin_lat2 + cos_lat1 * cos_lat2 * cos_delta_lng)

        return RADIUS * d


if __name__ == '__main__':

    import random
    random.seed(42)
    pts = [(random.uniform(-180, 180), random.uniform(-90, 90)) for _ in range(1000000)]
    
    centr = get_centroid(pts)

    import cProfile

    cProfile.runctx('gc_distance_points(centr, pts)', globals(), locals())
    cProfile.runctx('dists = [gc_dist(centr, b) for b in pts]', globals(), locals())

All code is also available at Github. Feel free to post a comment or email in case you have any questions.

Adding support for sftp to RCurl

2018-08-23T12:29:00.004+02:00

While trying to download data from an FTP with the sftp protocol from R I encountered the folllowing error on my mac:

Error in function (type, msg, asError = TRUE)  : 
  Protocol "sftp" not supported or disabled in libcurl

The easiest way to fix this for me was performing the following steps:

brew install curl --with-libssh2

Then check whether sftp is now among the list of supported protocols:

/usr/local/opt/curl/bin/curl -V

Link to the new version of curl you installed:

brew link --force curl

Re-install RCurl from source:

install.packages("RCurl", type = "source")

And finally check if sftp is now supported:

RCurl::curlVersion()

A new fizzbuzz: generating sums for kids

2018-05-23T20:34:00.002+02:00

With my oldest quizzing me for one sum after another for him to solve. I thought about writing a little piece of R code that would generate these for me. The code for doing so was less obvious then I imagined it to be.

Goal:

Generate all possible sums and subtractions for kids of the form

10 - 4 = ?
4 + 3 = ?

Constraints:

- Only numbers between 0-10
- Result should be >= 0

Solution:

generate_sums <- function(maxresult=10) {
  sums <- c()
  for (i in 0:(maxresult-1)) {
    for (j in 0:(maxresult-i)) {
      sums <- c(sums, paste0(i, ' + ', j, ' = ?'))
    }
  }
  sample(sums[!duplicated(sums)])
}
generate_subtractions <- function(maxresult=10) {
  substractions <- c()
  for (i in 0:maxresult) {
    for (j in i:maxresult) {
      substractions <- c(substractions, paste0(j, ' - ', i, ' = ?'))
    }
  }
  sample(substractions[!duplicated(substractions)])
}

I still believe that a more elegant approach should be possible but at least it works.

Fetching all records for a taxonomic group in OBIS

2018-02-28T21:21:00.000+01:00

For example, if you want to download the records for all Nudibranchia then in R it is as simple as doing:

    install.packages("robis")

    data <- robis::occurrence("Nudibranchia")

    # get species data
    spdata <- data[!is.na(data$taxonRank) & data$taxonRank == "species",]

    # plot on a map
    robis::leafletmap(spdata)

Alternatively, you can download all data with the new mapper that is being developed:

http://iobis.org/mapper2?scientificname=Nudibranchia&download=true

Error in linbin2D: (list) object cannot be coerced to type 'double'

2018-02-26T21:21:00.000+01:00

When calling bkde2D you might encounter the following error:

Error in linbin2D(x, gpoints1, gpoints2) :
(list) object cannot be coerced to type 'double'

This issue can manifest itself when the passed in data is a tibble instead of a data.frame.

Easy fix:

data <- as.data.frame(data)

Speeding up mean and standard deviation calculation for a stack of rasters in R

2018-02-24T21:21:00.000+01:00

While modelling the distribution of invasive seaweeds during my PhD I wanted to calculate the mean and standard deviation of a set of SDMs that appeared to be equivalent in for the current climate but who gave different results for the future climate. However, the standard naive approach for calculating the mean and standard deviation of a RasterStack was rather slow. I managed to get a 20 fold speed up by calculating the mean and standard deviation by aggregating them in a stream. This rather obvious for calculating the mean, summing all rasters and then dividing by the number of rasters. For the standard deviation, the formula is only slightly more complicated and can be found both on Wikipedia and in code on StackOverflow. If your raster contains very large numbers, you might need to adapt this code to use Welford's method.

# Option 1: shorter but slower

rasterstack_meansd_slow <- function(x) {
mean <- raster::mean(x)
  sd <- raster::calc(x, sd)

  list(mean=mean, sd=sd)

}

# Option 2: faster but more code
rasterstack_meansd_fast <- function(x) {
  s0 <- nlayers(x)
  s1 <- raster(x, layer=1)
  s2 <- s1^2
  for(ri in 2:s0) {
    r <- raster(x, layer=ri)
    s1 <- s1 + r
    s2 <- s2 + r^2
  }
  list(mean=s1/s0, sd=sqrt((s0 * s2 - s1 * s1)/(s0 * (s0 - 1))))
}

As a small example I calculate the mean and standard deviation of the sea surface temperature for different climate scenarios for 2050. The fast version only takes 10 seconds on my machine while the slow version takes 225 seconds to calculate the mean and standard deviation.

library(sdmpredictors)

sstfuture <- load_layers(c('BO2_RCP26_2050_tempmean_ss', 'BO2_RCP45_2050_tempmean_ss', 
                           'BO2_RCP60_2050_tempmean_ss', 'BO2_RCP85_2050_tempmean_ss'))

system.time({ fast <- rasterstack_meansd_fast(sstfuture) }) # 10 seconds
system.time({ slow <- rasterstack_meansd_slow(sstfuture) }) # 225 seconds

Selecting comma separated data as multiple rows with SQLite

2018-02-15T21:21:00.000+01:00

A while back I needed to split data stored in one column as a comma separated string into multiple rows in a SQL query from a SQLite database.

My table looked like this:

CREATE TABLE "predictor_sets" 
  (`id` INTEGER, `nvar` INTEGER, `predictors` TEXT, 
    `experiment` TEXT, PRIMARY KEY(`id`));

Insert some sample data:

INSERT INTO predictor_sets VALUES
  (1659, 5, 'BO_bathymax,BO_calcite,BO_parmax,BO_damean,BO_sstrange', 'bathymetry'),
  (1660, 5, 'BO_bathymin,BO_calcite,BO_parmax,BO_damean,BO_sstrange', 'bathymetry'),
  (1661, 5, 'BO_bathymean,BO_calcite,BO_parmax,BO_damean,BO_sstrange', 'bathymetry');

Splitting up the different predictors in seperate rows can be done with the following recursive common table expression:

WITH RECURSIVE split(predictorset_id, predictor_name, rest) AS (
  SELECT id, '', predictors || ',' FROM predictor_sets WHERE id
   UNION ALL
  SELECT predictorset_id, 
         substr(rest, 0, instr(rest, ',')),
         substr(rest, instr(rest, ',')+1)
    FROM split
   WHERE rest <> '')
SELECT predictorset_id, predictor_name 
  FROM split 
 WHERE predictor_name <> ''
 ORDER BY predictorset_id, predictor_name;

Check out the documentation for more info on writing your own common table expressions in SQLite, PostgreSQL or your favorite database.

Creating a single pixel wide raster along the coastline or another edge

2018-01-30T22:11:00.000+01:00

At one point during my PhD I needed a single pixel of raster values along the coastline. In order to get this I used the following code in R:

library(sdmpredictors)
library(raster)

coast_mask <- function(layer) {
  edges <- raster::boundaries(raster(layer, layer=1), type="inner")
  values <- getValues(edges)
  is.na(values) | values == 0
}

l <- load_layers("BO_sstmean", equalarea = FALSE)

mask <- coast_mask(l)
l[mask] <- NA

plot(l, col=rev(heat.colors(255)))

Source code available on GitHub at https://github.com/samuelbosch/blogbits/blob/master/misc/coastalraster.R

Main R Packages used:

Creating a high resolution png from a raster in R

2018-01-23T23:16:00.001+01:00

In this post I'll show you two different ways to create a png from an equal-area bathymetry grid in R.

Version 1: using the raster::plot function

The first option for plotting a raster is to use the raster::plot function and write this to a sufficiently large png file. This works reasonably well but the only disadvantage is that both the title and legend are really small. The title is optional but I didn't find a way to disable the legend. The main trick to get the raster::plot function to output a high resolution png consists in setting the maxpixels parameter. Colors where selected on ColorBrewer but with a small addition that the color range was reverted and that the darkest color is repeated to ensure that rare depths (-6000 to -10000) get the same color.

if(!requireNamespace("sdmpredictors")) {
  install.packages("sdmpredictors")
}
library(raster)
x <- sdmpredictors::load_layers("BO_bathymean", equalarea = TRUE)
png("bathymetry_plot1.png", width=ncol(x), height=nrow(x))
col <- rev(c("#f7fbff", "#deebf7", "#c6dbef", "#9ecae1", "#6baed6", "#4292c6","#2171b5","#08519c", rep("#08306b",7)))
plot(x, maxpixels = ncell(x), col = col, colNA = "#818181",

     main = "Bathymetry", axes = FALSE, ylim=extent(x)[3:4])
dev.off()

Version 2: write to png using leaflet colors

While this version looks and is a bit more complicated, it produces really good looking results. The main gist of this code is that it transforms the values to a range from 0 to 1000 while making sure that extreme values at both extremes will get the same colors as more common values. This is similar to what you general would do when e.g. creating a color scale with QGIS. Once values are mapped to colors they are converted to raw bytes with the right dimensions and written to a png file. Remark that this code was inspired by some of the internal functions in the leaflet package.

if(!requireNamespace("leaflet")) {
  install.packages("leaflet")
}
if(!requireNamespace("sdmpredictors")) {
  install.packages("sdmpredictors")
}
library(sdmpredictors)
library(raster)

# create colors
colors <- leaflet::colorNumeric(rev(c("#f7fbff", "#deebf7", "#c6dbef", "#9ecae1", "#6baed6", "#4292c6","#2171b5","#08519c", "#08306b")),
                                -1:1001, na.color =  "#818181")
cols <- c(colors(-1:1001), colors(NA))
x <- sdmpredictors::load_layers("BO_bathymean", equalarea = TRUE)
# scale values and remove extreme values from the color range 
vals <- values(x)
vals <- scale(vals)
minmax <- quantile(vals, probs=c(0.01, 0.99), na.rm = TRUE)
vals <- round((((vals - minmax[1]) / (minmax[2] - minmax[1])) * 1000))
vals[vals < 0] <- 0
vals[vals > 1000] <- 1000
vals[is.na(vals)] <- 1002

# lookup colors for scaled values, convert to raw and write to file
valcolors <- cols[vals+2] # +2 because -1 and 0 are in cols (value 0 is at index 2 in cols)

rgb_data <- col2rgb(valcolors, alpha = TRUE)
raw_data <- as.raw(rgb_data)
dim(raw_data) <- c(4, ncol(x), nrow(x))

png::writePNG(raw_data, "bathymetry_plot2.png")

All source code is available on GitHub at https://github.com/samuelbosch/blogbits/blob/master/misc/raster2png.R

Main R Packages used:

Intersecting a data frame with longitude/latitudes with a polygon shapefile in R

2017-01-10T09:59:00.001+01:00

I finally cleaned up a little script that I use to get the Spalding marine ecoregions for a set of points in R. The function needs as input a data frame with coordinates in longitude/latitude, the names of the coordinate columns (first the longitude column then the latitude column) and the path to the ecoregions shapefile. You can off course substitude this ecoregions shapefile with any other polygon shapefile (eg.countries, districts, lakes, protected areas, ...).

library(sp)
library(rgdal)

get_ecoregions <- function(data, xy_cols, meow_path) {
  df <- data
  row.names(df) <- 1:nrow(df)
  coordinates(data) <- xy_cols
  data@proj4string <-  CRS("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0")
  layer <- tools::file_path_sans_ext(basename(meow_path))
  meow <- readOGR(dsn = meow_path, layer = layer,verbose = FALSE, stringsAsFactors = FALSE)
  meow@proj4string <- data@proj4string # set the same, because they are
  intersect_meow_df <- gIntersection(meow, data, byid = TRUE)
  id_pairs <- strsplit(row.names(intersect_meow_df), " ")
  id_pairs_mat <- as.matrix(do.call(rbind, id_pairs))
  ecoids <- id_pairs_mat[,1]
  names(ecoids) <- id_pairs_mat[,2] ## store rowname from df.sp in names(ecoids)
  meow_row_names <- ecoids[row.names(data)]
  meow_row_names[is.na(meow_row_names)] <- "invalid row name"
  cbind(df, meow@data[meow_row_names,])
}

# df <- data.frame(species=rep("a", 4), longitude=c(4.087523,3.66044,2.926591, 0), latitude=c(51.671932, 51.55974, 51.234695,0))
# m <- get_ecoregions(df, c("longitude", "latitude"), "D:/a/data/ecoregions/meow_ecos.shp")

Compressing TIFF files in R and Windows

2016-09-07T23:11:00.000+02:00

Compressing TIFF files is an easy way to save disk space when working with raster files or image outputs for research papers. But this is rarely the default option causing used diskspace to quickly inflate heavily. For instance this figure 7000 by 5000 pixel image as a TIFF uses 136MB.

While after lossless compression with the libtiff command tiffcp it is only 8MB. To install libtiff on windows download and run the latest setup from http://gnuwin32.sourceforge.net/packages/tiff.htm. To compress a tiff file you need to run the following command (assuming that tiffcp is in your path). Make sure to specify a different output file name, otherwise the original will be overwritten with a broken and incomplete copy of the original.

tiffcp -c lzw uncompressed_input.tif compressed_output.tif

In R you can also write compressed raster (geo)tiff files by adding extra options to the writeRaster function from the raster package.

library(raster)

x <- raster("uncompressed_input.tif")
tifoptions <- c("COMPRESS=DEFLATE", "PREDICTOR=2", "ZLEVEL=6")
writeRaster(x, "compressed_output.tif",
            options = tifoptions, overwrite = FALSE)

Geographic null models for species distribution modeling: An implementation combining BIOMOD2 and dismo [Code Dump]

2016-09-07T22:22:00.001+02:00

Warning this is more of a code dump instead of a blogpost but I'm still putting it out there hoping it's still useful for someone else as it has been sitting in my draft folder for far too long. Anyways the paper by Robert Hijmans is really worth reading if you're interested in the evaluation of species distribution models. If you have questions, don't hesitate to contact me.

Geographic null model from paper by Robert Hijsmans (link to paper).
package dismo (maintained by Robert J. Hijmans). and the package BIOMOD2.

Load libraries:

library("biomod2")
library("dismo")

The null_model_evaluation function takes as input a BIOMOD2 model.

null_model_evaluation <- function(models) {
  ## flow
  ## 1) create geographic null model from the points used as training data
  ## 2) compare AUC of the model with AUC of the geographic null model (use same presence/absence data)
  
  ## extract test data from models
  data <- get_formal_data(models)
  calib.lines <- get(load(models@calib.lines@link))
  nbOfRuns <- (length(calib.lines) / length(data@data.species))
  calib.lines.mat <- as.matrix(calib.lines, nrow=length(data@data.species), ncol=nbOfRuns)

  evaluations <- get_evaluations(models)
  dims <- dimnames(evaluations)
  rocIndex <- which(dims[1][[1]] == "ROC")
  testingDataIndex <- which( dims[2][[1]] == "Testing.data")
  
  species.data <- data@coord[which(data@data.species == 1),]
  background.data <- data@coord[is.na(data@data.species),]
  
  result <- array(dim=c(nbOfRuns, 2, length(dims[3][[1]])), dimnames=list(dims[4][[1]],c("model", "geo_null_model"), dims[3][[1]]))
  
  ## calculate geographic null models
  for( methodIndex in 1:length(dims[3][[1]])){
    for (runIndex in 1:nbOfRuns){
      model.auc <- evaluations[rocIndex, testingDataIndex,methodIndex,runIndex,1]
      result[runIndex,1,methodIndex] <- model.auc
      
      train.species.data <- species.data[calib.lines[1:nrow(species.data),runIndex],]
      test.species.data <- species.data[!calib.lines[1:nrow(species.data),runIndex],]
      if (nrow(test.species.data) > 0 & !is.na(model.auc)){
        train.background.data <- background.data[calib.lines[nrow(species.data)+1:nrow(calib.lines)-nrow(species.data),runIndex],]
        test.background.data <- background.data[!calib.lines[nrow(species.data)+1:nrow(calib.lines)-nrow(species.data),runIndex],]
        gd <- geoDist(train.species.data, lonlat=TRUE)
        #p <- predict(gd, env)
        #plot(p)
        #points(train.background.data, pch=16, col="purple")
        #points(test.background.data, pch=16, col="blue")
        #points(train.species.data, pch=16, col="green")
        #points(test.species.data, pch=16, col="red")
        gd.evaluation <- evaluate(gd, p=test.species.data, a=test.background.data)
        result[runIndex,2,methodIndex] <- gd.evaluation@auc
      }
    }
  }
  result
}

Workaround for: SQL Expression Parsing Error: memory exhausted in QGIS or OGR

2016-08-31T15:11:00.000+02:00

Today I encountered an issue when I was trying to build a query with a lot of ids in an "IN" clause. I was trying to only show the polygons near Europe from the Global Self-consistent Hierarchical High-resolution Geography dataset (GSHHG) but thus wanted to only show more than 5500 id's.

With some find and replace actions I managed to built the following query:
"id" IN ('0-E','1','6','14','23','25','38','49','51','61','81','85','90','93','114','138','143','144','148','158','159','168','176','182','185','195','212','220','222','223','227','233','235','246','248','255','261','262','293','295','301','323','328','330','338','341','347','352','361','365','366','371','372','375','382','387','400','404','406','413','421','424','426','428','433','434','436','444','455','471','474','478','485','486','513','529','532','534','538','548','557','560','562','565','571','584','587','589','593','602','604','609','619','631','649','653','668','672','673','681','685','686','694','703','704','706','707','711','716','731','734','741','752','757','765','777','782','786','791','796','799','805','807','814','821','823','831','836','839','849','867','871','884','888','890','892','894','896','914','915','928','935','936','955','961','966','971','972','976','977','1000','1014','1035','1038','1046','1047','1049','1097','1100','1101','1105','1110','1115','1117','1120','1121','1122','1130','1133','1160','1174','1177','1187','1195','1209','1223','1229','1232','1243','1260','1261','1262','1268','1279','1291','1299','1306','1307','1310','1312','1315','1318','1323','1326','1332','1340','1351','1354','1372','1381','1392','1393','1396','1413','1418','1428','1432','1438','1445','1446','1449','1453','1461','1466','1482','1489','1501','1502','1505','1513','1520','1528','1532','1537','1539','1545','1547','1556','1560','1561','1564','1567','1570','1578','1584','1587','1592','1594','1595','1597','1603','1614','1622','1643','1655','1659','1660','1667','1680','1685','1687','1688','1696','1697','1701','1737','1756','1757','1767','1777','1789','1790','1797','1814','1815','1826','1827','1836','1840','1842','1843','1844','1846','1853','1863','1864','1867','1878','1881','1883','1889','1894','1899','1903','1926','1927','1928','1936','1948','1957','1964','1980','2003','2005','2018','2025','2026','2033','2043','2044','2055','2057','2062','2072','2073','2074','2079','2088','2089','2092','2095','2101','2107','2119','2131','2153','2156','2157','2162','2171','2176','2187','2190','2196','2198','2209','2242','2256','2265','2285','2287','2288','2290','2299','2318','2319','2324','2340','2342','2363','2365','2373','2374','2377','2383','2387','2395','2402','2413','2414','2421','2433','2441','2453','2467','2478','2481','2483','2487','2499','2503','2514','2533','2544','2546','2550','2556','2565','2567','2573','2574','2577','2582','2588','2589','2598','2603','2610','2614','2615','2620','2632','2640','2663','2664','2684','2714','2722','2725','2726','2748','2750','2758','2761','2764','2775','2778','2786','2788','2790','2806','2814','2815','2817','2820','2829','2834','2837','2856','2875','2881','2898','2902','2908','2932','2948','2949','2955','2959','2969','2973','2979','3000','3001','3003','3009','3015','3018','3019','3038','3055','3057','3071','3081','3082','3092','3093','3094','3117','3119','3127','3132','3133','3140','3160','3165','3184','3188','3195','3200','3205','3215','3216','3226','3229','3230','3236','3255','3256','3259','3260','3287','3288','3297','3323','3325','3329','3331','3340','3345','3358','3365','3372','3376','3384','3388','3404','3405','3421','3423','3430','3450','3460','3468','3476','3477','3483','3487','3497','3499','3500','3508','3514','3515','3516','3527','3532','3533','3555','3576','3585','3602','3603','3631','3641','3643','3646','3648','3649','3651','3652','3658','3671','3673','3681','3687','3688','3693','3702','3713','3716','3719','3723','3762','3765','3769','3777','3786','3793','3802','3803','3807','3808','3814','3816','3821','3826','3829','3831','3837','3839','3841','3842','3845','3852','3855','3856','3860','3865','3869','3872','3877','3888','3889','3899','3903','3905','3916','3918','3926','3929','3941','3947','3950','3953','3962','3970','3982','4015','4018','4030','4031','4040','4041','4044','4050','4055','4069','4071','4072','4076','4084','4085','4087','4093','4095','4106','4109','4113','4118','4119','4121','4124','4125','4129','4153','4156','4165','4174','4180','4192','4208','4214','4220','4225','4228','4231','4240','4268','4277','4281','4294','4295','4297','4303','4318','4324','4325','4326','4327','4337','4343','4345','4351','4354','4359','4364','4365','4388','4392','4398','4404','4405','4424','4431','4445','4458','4466','4468','4470','4472','4480','4487','4492','4493','4498','4499','4504','4510','4513','4521','4531','4543','4546','4549','4552','4556','4557','4558','4561','4562','4576','4583','4586','4597','4614','4617','4627','4646','4649','4654','4664','4668','4714','4723','4728','4752','4760','4761','4763','4767','4772','4791','4814','4823','4825','4840','4841','4847','4850','4865','4873','4884','4893','4895','4905','4918','4920','4935','4936','4943','4953','4964','4969','4978','4979','4980','4994','5003','5015','5017','5019','5021','5028','5030','5033','5039','5041','5042','5046','5050','5052','5053','5062','5064','5072','5079','5083','5087','5098','5111','5113','5119','5121','5147','5148','5154','5157','5159','5178','5185','5186','5192','5193','5201','5223','5238','5240','5251','5253','5260','5280','5288','5296','5297','5305','5306','5310','5313','5316','5319','5327','5329','5346','5347','5348','5362','5370','5371','5372','5382','5400','5411','5416','5427','5433','5438','5440','5444','5447','5453','5458','5459','5461','5465','5472','5474','5475','5483','5487','5494','5504','5507','5527','5536','5539','5542','5548','5561','5563','5564','5569','5571','5581','5584','5587','5588','5604','5609','5610','5614','5635','5642','5652','5653','5662','5669','5673','5678','5683','5691','5693','5695','5699','5704','5730','5732','5735','5743','5748','5758','5765','5766','5768','5770','5774','5779','5792','5811','5814','5819','5829','5830','5831','5856','5862','5875','5876','5879','5883','5885','5899','5903','5907','5908','5913','5921','5934','5941','5948','5953','5960','5964','5969','5973','5981','5985','5992','5993','5996','5998','6002','6004','6008','6015','6018','6020','6028','6034','6036','6048','6049','6059','6067','6073','6081','6084','6086','6093','6114','6122','6130','6137','6138','6142','6143','6157','6163','6165','6167','6178','6187','6192','6195','6201','6205','6206','6209','6218','6230','6245','6254','6260','6264','6275','6280','6281','6284','6305','6308','6312','6325','6326','6334','6359','6360','6362','6367','6373','6379','6389','6390','6394','6403','6417','6420','6423','6438','6443','6451','6459','6463','6466','6468','6470','6477','6482','6491','6500','6505','6509','6514','6518','6520','6525','6528','6541','6544','6547','6551','6559','6574','6585','6589','6594','6606','6609','6619','6627','6661','6668','6670','6671','6674','6680','6688','6696','6699','6707','6713','6718','6724','6745','6751','6755','6773','6781','6798','6810','6812','6844','6859','6860','6877','6878','6879','6888','6889','6891','6893','6901','6909','6912','6925','6926','6932','6935','6942','6943','6950','6957','6970','6976','6981','6987','6989','6990','6991','6997','7001','7005','7026','7027','7035','7048','7053','7057','7059','7061','7064','7080','7083','7089','7090','7091','7092','7094','7096','7104','7106','7108','7111','7112','7115','7129','7137','7144','7155','7157','7167','7177','7180','7200','7202','7214','7232','7235','7238','7248','7252','7256','7268','7272','7287','7292','7301','7322','7324','7326','7331','7333','7346','7349','7356','7359','7361','7371','7376','7377','7387','7388','7394','7401','7426','7427','7432','7435','7437','7438','7445','7446','7451','7454','7468','7485','7486','7514','7517','7536','7540','7544','7551','7555','7557','7572','7577','7589','7604','7606','7626','7627','7633','7642','7643','7652','7654','7668','7690','7696','7697','7702','7717','7721','7743','7746','7748','7749','7750','7751','7760','7764','7765','7772','7777','7779','7808','7813','7828','7863','7870','7876','7884','7889','7897','7898','7900','7903','7911','7912','7919','7922','7925','7934','7936','7944','7945','7947','7962','7964','7969','7975','7978','7990','7994','8000','8015','8023','8028','8035','8040','8041','8048','8051','8055','8056','8057','8062','8071','8080','8083','8086','8097','8102','8106','8113','8120','8135','8139','8152','8164','8172','8174','8175','8178','8182','8191','8199','8205','8210','8211','8216','8221','8223','8224','8232','8240','8241','8252','8275','8292','8303','8306','8307','8321','8326','8335','8350','8358','8365','8374','8376','8386','8387','8396','8398','8404','8411','8433','8443','8456','8458','8479','8482','8489','8496','8499','8504','8511','8521','8523','8524','8529','8530','8534','8538','8548','8549','8555','8558','8561','8562','8571','8579','8589','8593','8606','8614','8628','8629','8638','8640','8653','8661','8669','8676','8694','8698','8720','8723','8729','8733','8742','8745','8752','8756','8758','8763','8774','8791','8795','8809','8816','8820','8821','8836','8837','8838','8840','8845','8849','8851','8853','8856','8865','8867','8877','8878','8892','8901','8906','8912','8914','8915','8923','8941','8946','8956','8972','8981','8986','8991','8992','9007','9023','9026','9039','9041','9049','9074','9077','9086','9096','9101','9106','9114','9120','9132','9135','9137','9138','9159','9161','9164','9183','9189','9204','9210','9217','9220','9226','9228','9231','9236','9237','9239','9243','9245','9248','9250','9260','9262','9264','9274','9277','9299','9310','9311','9314','9316','9336','9356','9365','9367','9368','9370','9374','9378','9393','9398','9411','9415','9439','9440','9452','9468','9470','9474','9476','9477','9484','9493','9507','9516','9517','9534','9538','9542','9554','9557','9569','9570','9579','9587','9602','9613','9626','9636','9672','9691','9713','9725','9739','9744','9747','9754','9758','9760','9763','9765','9772','9775','9778','9779','9792','9801','9818','9826','9864','9871','9875','9876','9920','9921','9923','9929','9939','9943','9949','9953','9957','9971','9984','9991','10007','10010','10017','10020','10026','10027','10034','10046','10053','10055','10058','10060','10061','10063','10067','10077','10078','10095','10101','10106','10132','10133','10135','10140','10149','10150','10157','10160','10163','10181','10190','10192','10208','10221','10226','10243','10251','10256','10271','10274','10279','10289','10296','10305','10314','10348','10368','10372','10373','10379','10388','10403','10417','10428','10429','10432','10433','10435','10436','10442','10444','10449','10452','10453','10456','10460','10472','10495','10504','10513','10540','10546','10551','10561','10580','10586','10587','10589','10599','10612','10632','10642','10644','10645','10647','10660','10679','10681','10682','10684','10685','10687','10692','10693','10697','10709','10715','10720','10738','10745','10748','10751','10760','10762','10773','10788','10797','10798','10800','10819','10820','10826','10830','10859','10860','10863','10867','10877','10884','10895','10896','10902','10909','10910','10911','10912','10914','10916','10918','10921','10924','10937','10941','10968','10980','10981','10984','10987','10988','10993','10997','11003','11023','11026','11051','11056','11066','11077','11082','11086','11091','11092','11102','11108','11110','11118','11128','11138','11141','11146','11174','11177','11180','11194','11195','11205','11208','11213','11217','11220','11224','11243','11244','11253','11255','11262','11270','11272','11285','11287','11296','11297','11301','11307','11308','11318','11321','11339','11346','11356','11364','11366','11370','11375','11377','11380','11404','11405','11410','11414','11417','11446','11448','11451','11459','11464','11476','11479','11502','11512','11515','11537','11541','11542','11544','11551','11563','11565','11578','11581','11585','11592','11595','11599','11600','11605','11618','11630','11637','11643','11648','11652','11654','11657','11664','11667','11671','11674','11677','11678','11688','11698','11699','11701','11705','11707','11710','11721','11724','11728','11736','11751','11757','11758','11762','11771','11780','11783','11784','11796','11798','11801','11809','11813','11821','11826','11844','11845','11848','11854','11858','11863','11864','11867','11871','11874','11891','11892','11894','11897','11902','11907','11910','11912','11914','11917','11943','11947','11950','11958','11967','11999','12023','12028','12036','12039','12060','12066','12074','12076','12080','12082','12086','12087','12096','12106','12115','12120','12126','12136','12139','12143','12150','12155','12159','12164','12182','12193','12195','12196','12205','12213','12238','12242','12244','12253','12257','12271','12281','12284','12292','12295','12301','12318','12329','12337','12338','12352','12360','12363','12364','12374','12382','12401','12432','12438','12440','12443','12444','12455','12459','12461','12464','12486','12491','12498','12510','12513','12514','12534','12537','12539','12540','12551','12564','12567','12573','12574','12581','12584','12586','12600','12603','12611','12615','12625','12629','12638','12642','12656','12659','12661','12671','12677','12678','12684','12692','12693','12695','12697','12716','12724','12735','12756','12775','12787','12790','12805','12809','12824','12835','12837','12844','12845','12857','12861','12868','12876','12880','12881','12884','12893','12903','12908','12923','12926','12935','12940','12947','12953','12956','12965','12977','12979','12980','12984','12992','12996','12998','13002','13012','13027','13062','13066','13081','13087','13100','13103','13106','13107','13117','13120','13123','13131','13146','13148','13149','13154','13160','13162','13166','13170','13184','13185','13186','13190','13192','13193','13196','13215','13216','13218','13219','13230','13238','13239','13244','13247','13253','13259','13265','13266','13269','13282','13285','13286','13289','13300','13301','13307','13312','13313','13318','13320','13331','13332','13357','13358','13366','13367','13375','13389','13407','13408','13409','13410','13416','13429','13433','13438','13440','13459','13465','13470','13475','13481','13482','13483','13486','13487','13490','13502','13505','13506','13510','13514','13527','13540','13545','13546','13547','13564','13565','13566','13570','13572','13576','13578','13580','13581','13583','13585','13591','13593','13595','13597','13601','13606','13611','13619','13624','13631','13635','13637','13638','13639','13651','13660','13678','13683','13686','13726','13729','13732','13737','13738','13741','13742','13756','13759','13762','13765','13769','13770','13775','13784','13792','13805','13808','13820','13827','13834','13841','13855','13857','13862','13868','13871','13881','13882','13889','13891','13896','13905','13911','13913','13926','13929','13932','13942','13962','13989','13990','13994','13995','13997','14003','14004','14006','14007','14010','14013','14015','14017','14019','14024','14026','14030','14033','14034','14069','14075','14078','14080','14081','14082','14085','14089','14090','14091','14093','14097','14099','14106','14107','14113','14116','14135','14147','14149','14153','14158','14160','14168','14169','14175','14176','14190','14192','14204','14206','14207','14218','14223','14231','14248','14249','14252','14258','14261','14266','14267','14272','14274','14280','14281','14286','14295','14297','14308','14319','14322','14324','14327','14328','14330','14332','14336','14337','14341','14351','14363','14369','14373','14376','14377','14378','14380','14381','14386','14390','14393','14398','14410','14417','14436','14438','14445','14447','14450','14451','14454','14457','14466','14467','14469','14472','14474','14480','14481','14485','14486','14495','14504','14509','14523','14526','14538','14555','14577','14595','14596','14602','14610','14611','14641','14646','14651','14652','14657','14688','14700','14707','14709','14722','14726','14727','14730','14735','14744','14750','14761','14766','14770','14778','14789','14795','14803','14804','14813','14820','14826','14837','14842','14849','14854','14856','14862','14874','14878','14879','14880','14909','14925','14931','14936','14945','14961','14966','14968','14975','14989','14991','14996','14999','15013','15024','15025','15047','15049','15060','15061','15064','15074','15088','15092','15105','15111','15112','15118','15119','15121','15123','15128','15135','15143','15155','15156','15164','15168','15172','15173','15175','15177','15181','15192','15196','15201','15203','15212','15213','15214','15216','15224','15229','15234','15235','15236','15250','15253','15255','15261','15262','15267','15274','15277','15287','15320','15321','15329','15331','15332','15338','15340','15341','15352','15356','15359','15369','15379','15381','15386','15390','15394','15399','15406','15412','15413','15423','15428','15433','15442','15448','15455','15456','15460','15472','15473','15491','15494','15512','15529','15538','15559','15570','15582','15586','15588','15591','15592','15599','15603','15605','15606','15610','15613','15631','15632','15656','15664','15665','15670','15671','15677','15685','15697','15703','15719','15725','15731','15736','15739','15745','15746','15752','15753','15759','15770','15773','15775','15778','15782','15785','15786','15792','15796','15801','15804','15809','15810','15815','15817','15821','15827','15832','15833','15849','15855','15858','15866','15874','15879','15884','15894','15896','15907','15908','15915','15916','15930','15943','15952','15955','15960','15962','15973','15976','15989','15995','16002','16003','16006','16007','16017','16023','16029','16047','16049','16058','16065','16075','16092','16101','16107','16108','16123','16138','16141','16144','16155','16159','16171','16188','16192','16200','16201','16205','16208','16209','16214','16215','16218','16220','16221','16223','16224','16232','16242','16243','16252','16255','16261','16266','16267','16270','16271','16272','16276','16277','16288','16290','16299','16303','16305','16306','16315','16336','16342','16349','16355','16362','16371','16377','16378','16386','16390','16391','16434','16442','16448','16452','16462','16463','16466','16468','16491','16500','16502','16504','16519','16524','16536','16539','16542','16549','16555','16561','16565','16570','16576','16579','16594','16599','16603','16604','16605','16622','16623','16629','16635','16639','16640','16649','16657','16659','16662','16663','16666','16670','16671','16679','16687','16694','16696','16698','16700','16715','16723','16731','16733','16739','16743','16754','16755','16758','16762','16777','16800','16803','16806','16809','16828','16845','16867','16870','16882','16883','16891','16906','16909','16912','16913','16919','16922','16925','16926','16945','16947','16949','16967','16978','16981','16987','16990','16998','17001','17011','17012','17022','17024','17025','17028','17038','17040','17041','17042','17048','17050','17056','17061','17067','17070','17073','17093','17094','17105','17107','17110','17112','17114','17118','17127','17135','17149','17156','17165','17168','17171','17175','17187','17188','17193','17194','17197','17207','17209','17211','17218','17219','17221','17222','17226','17229','17240','17247','17251','17253','17276','17283','17292','17295','17302','17304','17315','17324','17326','17329','17331','17336','17347','17351','17353','17362','17363','17370','17377','17386','17388','17390','17399','17403','17404','17405','17407','17408','17411','17419','17435','17437','17441','17451','17463','17475','17482','17484','17485','17491','17493','17496','17503','17505','17507','17509','17520','17521','17529','17534','17535','17547','17558','17560','17568','17570','17579','17584','17587','17596','17598','17600','17601','17602','17605','17616','17632','17639','17648','17649','17652','17656','17660','17680','17681','17684','17691','17700','17701','17706','17710','17724','17729','17741','17742','17747','17750','17757','17762','17774','17775','17782','17786','17790','17804','17813','17830','17831','17832','17833','17835','17837','17842','17843','17845','17846','17847','17851','17854','17858','17861','17863','17866','17887','17897','17914','17919','17921','17922','17933','17941','17950','17961','17968','17974','17983','17988','17989','17998','17999','18000','18006','18019','18025','18035','18036','18049','18050','18058','18069','18077','18078','18092','18098','18103','18105','18114','18118','18119','18125','18127','18136','18138','18144','18149','18153','18158','18163','18166','18176','18181','18184','18192','18196','18209','18228','18229','18235','18236','18246','18269','18272','18283','18288','18290','18293','18295','18312','18326','18332','18335','18337','18343','18345','18353','18358','18364','18380','18382','18396','18402','18403','18417','18423','18427','18432','18435','18441','18454','18457','18461','18481','18488','18489','18494','18495','18502','18506','18518','18535','18536','18549','18554','18559','18595','18596','18612','18623','18625','18628','18629','18634','18637','18642','18645','18656','18662','18664','18673','18687','18693','18702','18715','18716','18719','18720','18725','18730','18742','18743','18746','18748','18755','18757','18772','18775','18776','18781','18787','18788','18790','18797','18809','18811','18813','18816','18823','18825','18851','18858','18871','18872','18875','18891','18903','18906','18907','18908','18910','18913','18915','18920','18937','18941','18942','18943','18946','18948','18949','18954','18957','18958','18962','18964','18968','18976','18981','18988','18990','18996','19005','19014','19019','19023','19031','19037','19040','19042','19058','19068','19070','19074','19079','19086','19091','19094','19095','19105','19110','19112','19115','19121','19122','19123','19136','19145','19157','19165','19169','19171','19176','19181','19186','19189','19194','19197','19199','19207','19208','19214','19223','19225','19228','19237','19249','19252','19255','19264','19268','19269','19270','19274','19295','19301','19307','19308','19314','19317','19318','19321','19323','19324','19335','19337','19339','19343','19348','19349','19353','19358','19364','19371','19382','19384','19385','19395','19398','19402','19405','19422','19430','19447','19448','19453','19454','19456','19465','19466','19476','19478','19484','19486','19488','19490','19501','19506','19514','19519','19521','19522','19527','19539','19540','19553','19555','19557','19564','19577','19579','19580','19590','19604','19612','19614','19623','19631','19634','19642','19648','19651','19654','19655','19656','19668','19677','19682','19686','19695','19721','19724','19730','19749','19751','19753','19755','19759','19762','19766','19767','19768','19772','19778','19791','19807','19812','19818','19829','19836','19838','19843','19847','19848','19850','19852','19853','19854','19858','19861','19868','19870','19878','19884','19910','19914','19918','19921','19922','19925','19928','19942','19943','19947','19950','19959','19971','19977','19981','19985','19986','19994','20000','20003','20007','20008','20015','20028','20029','20041','20056','20063','20069','20075','20078','20083','20085','20089','20091','20098','20105','20128','20129','20143','20146','20153','20154','20156','20159','20170','20180','20181','20182','20183','20188','20189','20195','20200','20204','20209','20216','20218','20221','20224','20229','20233','20234','20237','20239','20241','20253','20256','20260','20263','20264','20265','20268','20271','20274','20289','20290','20291','20304','20306','20307','20308','20311','20317','20324','20339','20348','20350','20357','20363','20369','20370','20376','20379','20387','20388','20403','20404','20407','20413','20415','20425','20433','20437','20440','20442','20443','20444','20446','20448','20450','20458','20464','20481','20489','20495','20496','20511','20512','20513','20514','20517','20522','20534','20535','20540','20541','20550','20555','20556','20558','20561','20569','20570','20573','20582','20613','20614','20623','20631','20637','20643','20647','20669','20686','20695','20696','20705','20707','20716','20718','20724','20728','20729','20736','20738','20742','20748','20761','20779','20782','20785','20793','20812','20814','20840','20846','20848','20853','20857','20872','20875','20880','20881','20885','20891','20899','20914','20917','20924','20937','20943','20944','20947','20959','20970','20974','20982','20984','20985','20986','21003','21006','21013','21028','21032','21036','21047','21054','21056','21057','21058','21079','21087','21094','21097','21102','21111','21113','21118','21120','21131','21138','21153','21154','21167','21168','21170','21172','21186','21187','21191','21202','21213','21216','21217','21220','21227','21238','21241','21247','21249','21271','21281','21290','21300','21304','21306','21312','21316','21318','21320','21328','21334','21337','21338','21348','21350','21352','21356','21399','21403','21423','21424','21435','21441','21449','21455','21463','21478','21481','21482','21486','21487','21491','21494','21509','21510','21512','21518','21521','21523','21529','21530','21532','21533','21535','21537','21549','21577','21578','21579','21583','21587','21598','21604','21613','21623','21632','21641','21643','21652','21655','21656','21659','21671','21673','21674','21683','21684','21690','21697','21713','21716','21721','21725','21727','21733','21734','21735','21737','21745','21746','21751','21753','21758','21775','21779','21788','21789','21793','21795','21800','21809','21813','21814','21826','21831','21850','21858','21862','21869','21873','21875','21877','21880','21886','21892','21893','21896','21898','21900','21903','21917','21919','21923','21926','21934','21936','21938','21940','21950','21952','21961','21968','21972','21978','21985','21994','22002','22008','22009','22011','22014','22015','22017','22019','22026','22032','22045','22065','22067','22079','22088','22096','22105','22107','22109','22111','22116','22123','22136','22146','22150','22161','22176','22184','22196','22201','22209','22212','22215','22217','22223','22224','22229','22232','22234','22235','22236','22249','22252','22255','22259','22262','22268','22269','22272','22276','22278','22292','22308','22310','22312','22315','22316','22326','22338','22340','22341','22342','22347','22351','22368','22374','22386','22389','22393','22397','22399','22400','22412','22414','22421','22426','22441','22442','22443','22446','22449','22465','22467','22490','22492','22494','22497','22499','22506','22508','22512','22516','22519','22522','22527','22548','22552','22554','22559','22569','22574','22575','22576','22581','22583','22586','22593','22604','22609','22610','22617','22623','22625','22633','22634','22637','22638','22640','22642','22645','22647','22651','22661','22662','22665','22666','22677','22678','22681','22682','22683','22685','22694','22695','22701','22702','22708','22713','22724','22726','22729','22731','22741','22745','22747','22748','22761','22782','22784','22795','22796','22798','22800','22804','22823','22833','22835','22844','22847','22850','22854','22857','22858','22864','22872','22875','22883','22891','22902','22904','22911','22934','22954','22970','22972','22974','22983','22993','23007','23009','23012','23013','23017','23023','23037','23046','23047','23048','23049','23061','23072','23083','23091','23109','23113','23114','23115','23116','23125','23127','23128','23133','23140','23143','23145','23147','23154','23160','23173','23180','23182','23186','23187','23204','23207','23211','23215','23217','23223','23224','23239','23257','23266','23268','23275','23277','23282','23287','23288','23289','23290','23301','23302','23306','23314','23315','23319','23322','23331','23336','23341','23344','23345','23348','23350','23354','23355','23365','23368','23369','23371','23372','23375','23381','23387','23394','23395','23398','23408','23415','23435','23438','23448','23453','23454','23462','23465','23466','23478','23481','23486','23490','23491','23498','23516','23519','23528','23536','23537','23541','23544','23549','23550','23553','23554','23555','23557','23560','23568','23569','23570','23578','23587','23588','23595','23597','23602','23605','23613','23615','23619','23620','23626','23627','23631','23638','23648','23654','23655','23658','23665','23668','23680','23700','23707','23711','23718','23722','23723','23725','23727','23731','23734','23740','23747','23759','23764','23766','23784','23786','23790','23794','23802','23811','23814','23819','23821','23827','23830','23834','23843','23846','23861','23863','23865','23867','23870','23875','23896','23898','23899','23910','23911','23912','23920','23938','23939','23946','23949','23971','23972','23975','23977','23981','23984','23987','23997','24002','24003','24009','24012','24023','24024','24031','24034','24043','24048','24051','24056','24062','24070','24074','24077','24079','24091','24094','24095','24109','24110','24113','24116','24122','24125','24128','24129','24136','24141','24148','24150','24152','24160','24161','24162','24163','24166','24170','24171','24172','24175','24179','24181','24183','24196','24210','24211','24225','24232','24241','24244','24245','24249','24250','24258','24262','24267','24270','24272','24274','24279','24295','24303','24304','24313','24320','24321','24326','24328','24332','24335','24340','24347','24349','24354','24355','24356','24358','24362','24365','24367','24373','24374','24377','24386','24389','24405','24407','24426','24430','24432','24441','24445','24447','24448','24473','24479','24485','24486','24490','24495','24496','24506','24512','24514','24516','24526','24536','24540','24544','24546','24565','24569','24576','24578','24582','24583','24584','24585','24587','24592','24595','24599','24600','24603','24608','24618','24624','24625','24626','24633','24639','24641','24645','24649','24665','24668','24673','24678','24679','24687','24689','24695','24696','24699','24705','24708','24711','24714','24715','24716','24719','24720','24721','24747','24751','24755','24780','24784','24787','24789','24792','24797','24817','24824','24825','24826','24827','24828','24837','24845','24858','24869','24875','24892','24896','24899','24907','24908','24909','24910','24911','24913','24914','24916','24921','24933','24935','24953','24954','24955','24958','24965','24968','24994','25002','25012','25014','25016','25021','25028','25032','25051','25054','25056','25057','25061','25076','25085','25089','25091','25092','25097','25098','25107','25112','25114','25116','25117','25125','25128','25133','25137','25142','25144','25146','25148','25159','25160','25166','25167','25170','25180','25181','25185','25186','25197','25198','25203','25204','25211','25227','25229','25232','25235','25239','25242','25244','25245','25259','25261','25263','25268','25272','25282','25284','25289','25290','25291','25294','25297','25300','25303','25306','25322','25323','25326','25332','25333','25337','25339','25349','25351','25352','25354','25358','25360','25362','25394','25395','25398','25401','25402','25403','25408','25409','25416','25429','25435','25450','25451','25452','25458','25459','25486','25491','25495','25496','25503','25505','25508','25513','25514','25519','25520','25522','25524','25526','25528','25530','25534','25535','25544','25556','25558','25560','25563','25571','25572','25578','25581','25586','25595','25598','25599','25600','25601','25617','25622','25631','25638','25640','25641','25644','25648','25649','25650','25652','25654','25660','25669','25673','25676','25699','25700','25701','25705','25708','25721','25723','25730','25735','25739','25745','25755','25765','25770','25773','25777','25782','25797','25800','25803','25820','25825','25829','25830','25835','25848','25856','25858','25859','25861','25869','25875','25881','25885','25890','25891','25896','25897','25899','25911','25913','25921','25922','25923','25925','25926','25932','25934','25940','25942','25946','25954','25959','25960','25971','25983','25985','26012','26015','26018','26024','26034','26041','26049','26053','26055','26058','26062','26074','26079','26090','26098','26118','26123','26141','26154','26155','26156','26159','26163','26166','26182','26184','26185','26187','26190','26198','26205','26212','26217','26228','26229','26230','26232','26237','26239','26243','26252','26253','26254','26255','26257','26263','26264','26265','26272','26273','26284','26291','26297','26324','26335','26338','26346','26352','26361','26371','26375','26376','26379','26380','26384','26391','26394','26395','26403','26404','26412','26420','26425','26432','26434','26436','26438','26444','26447','26449','26472','26478','26481','26497','26498','26500','26501','26508','26513','26514','26521','26523','26524','26525','26533','26536','26549','26555','26558','26560','26583','26585','26586','26598','26613','26617','26623','26633','26649','26652','26675','26676','26678','26680','26686','26692','26694','26696','26700','26703','26706','26710','26712','26716','26717','26718','26720','26721','26724','26728','26730','26733','26746','26754','26758','26759','26766','26767','26769','26781','26790','26797','26802','26815','26818','26824','26830','26837','26842','26856','26858','26864','26872','26874','26875','26876','26882','26883','26885','26887','26889','26894','26895','26930','26937','26939','26945','26960','26976','26979','26980','26983','26985','26990','27002','27011','27012','27014','27034','27036','27037','27046','27048','27053','27058','27064','27088','27094','27106','27111','27118','27121','27122','27127','27131','27133','27137','27142','27144','27145','27146','27148','27150','27152','27153','27154','27158','27165','27166','27172','27173','27175','27179','27189','27194','27197','27203','27206','27225','27229','27242','27246','27256','27272','27274','27286','27288','27297','27303','27313','27316','27317','27319','27322','27323','27324','27325','27327','27330','27334','27354','27359','27365','27367','27381','27382','27383','27388','27395','27396','27400','27407','27408','27411','27415','27422','27428','27429','27437','27442','27447','27453','27455','27457','27467','27470','27479','27487','27489','27490','27499','27506','27511','27513','27516','27517','27521','27525','27534','27539','27546','27549','27554','27555','27558','27560','27566','27573','27574','27582','27589','27593','27595','27601','27607','27612','27623','27626','27630','27631','27637','27638','27641','27642','27644','27645','27646','27653','27655','27662','27663','27681','27684','27693','27696','27698','27707','27708','27711','27715','27719','27722','27724','27728','27729','27739','27750','27756','27768','27769','27773','27777','27785','27791','27800','27803','27804','27808','27809','27810','27813','27819','27821','27824','27830','27844','27845','27846','27853','27857','27870','27872','27882','27888','27891','27900','27907','27908','27914','27915','27917','27925','27927','27930','27932','27933','27935','27936','27937','27939','27940','27941','27943','27949','27955','27960','27962','27966','27969','27976','27978','27986','28009','28013','28017','28026','28028','28030','28032','28033','28036','28041','28044','28045','28047','28051','28058','28063','28064','28066','28068','28069','28070','28073','28083','28085','28090','28094','28097','28100','28109','28111','28118','28119','28120','28132','28138','28142','28143','28147','28158','28162','28164','28169','28174','28175','28181','28186','28189','28190','28192','28195','28201','28214','28224','28226','28227','28231','28245','28251','28258','28262','28270','28274','28284','28290','28292','28298','28307','28323','28327','28330','28337','28339','28340','28346','28375','28376','28384','28386','28389','28394','28395','28398','28414','28415','28421','28424','28434','28444','28455','28464','28473','28476','28479','28481','28483','28484','28485','28490','28491','28494','28508','28509','28523','28524','28526','28545','28548','28549','28556','28558','28568','28572','28578','28579','28582','28590','28591','28596','28600','28603','28613','28616','28618','28620','28623','28634','28635','28640','28641','28643','28651','28659','28660','28661','28665','28668','28669','28680','28686','28693','28694','28695','28700','28701','28703','28704','28711','28712','28720','28724','28736','28740','28741','28742','28746','28747','28751','28756','28758','28767','28769','28773','28775','28777','28778','28791','28808','28817','28819','28822','28823','28831','28835','28841','28845','28847','28850','28852','28856','28862','28865','28867','28869','28872','28880','28897','28899','28901','28907','28908','28913','28914','28915','28923','28926','28934','28939','28940','28945','28947','28948','28949','28964','28967','28972','28987','28990','29002','29005','29012','29018','29026','29033','29035','29038','29039','29040','29054','29058','29065','29069','29070','29079','29083','29090','29095','29108','29109','29112','29115','29120','29125','29127','29136','29139','29141','29143','29145','29146','29157','29162','29163','29178','29190','29193','29200','29207','29208','29212','29222','29223','29227','29229','29235','29236','29238','29242','29250','29252','29260','29266','29271','29273','29281','29282','29289','29295','29296','29301','29305','29306','29308','29310','29314','29316','29323','29329','29332','29335','29336','29338','29344','29346','29347','29351','29352','29356','29360','29364','29369','29370','29375','29376','29382','29391','29394','29406','29407','29410','29423','29437','29441','29444','29446','29449','29453','29454','29468','29469','29471','29474','29477','29479','29480','29481','29487','29504','29510','29516','29522','29527','29537','29538','29539','29543','29552','29554','29555','29556','29562','29569','29571','29575','29576','29579','29587','29592','29594','29595','29615','29622','29623','29624','29625','29630','29631','29632','29635','29640','29642','29647','29648','29650','29659','29663','29671','29675','29684','29693','29694','29700','29701','29706','29722','29729','29731','29732','29735','29740','29751','29757','29759','29761','29763','29769','29770','29771','29779','29780','29783','29785','29787','29788','29793','29794','29795','29797','29808','29811','29813','29831','29833','29839','29842','29847','29848','29851','29866','29867','29870','29877','29881','29883','29884','29893','29898','29905','29908','29911','29914','29916','29918','29920','29926','29939','29953','29956','29965','29978','29985','29992','29997','30009','30011','30017','30026','30027','30029','30031','30034','30040','30044','30045','30047','30048','30054','30063','30064','30068','30071','30074','30077','30079','30081','30096','30098','30101','30104','30111','30116','30129','30131','30141','30147','30156','30160','30162','30169','30171','30175','30178','30185','30186','30191','30192','30193','30198','30201','30203','30204','30209','30220','30222','30233','30239','30246','30248','30249','30255','30257','30258','30260','30269','30271','30274','30279','30280','30281','30283','30287','30288','30296','30300','30308','30309','30324','30328','30331','30333','30335','30339','30341','30343','30344','30346','30348','30352','30356','30358','30362','30372','30375','30378','30379','30388','30394','30400','30404','30422','30435','30440','30445','30447','30449','30458','30462','30474','30480','30488','30489','30500','30501','30503','30504','30506','30507','30509','30516','30521','30524','30528','30529','30538','30540','30548','30549','30558','30560','30562','30575','30583','30587','30591','30595','30596','30598','30605','30606','30609','30610','30611','30613','30618','30627','30628','30629','30635','30641','30645','30646','30659','30660','30669','30670','30680','30682','30683','30690','30692','30696','30697','30701','30703','30704','30708','30715','30717','30733','30740','30749','30751','30754','30760','30762','30765','30766','30768','30770','30772','30777','30784','30787','30792','30799','30807','30808','30811','30815','30817','30818','30822','30833','30836','30837','30852','30853','30854','30864','30870','30873','30881','30883','30885','30888','30890','30896','30904','30905','30908','30909','30916','30920','30925','30926','30927','30935','30936','30943','30955','30956','30962','30971','30974','30975','30979','30981','30982','30983','30988','30989','31006','31010','31022','31025','31032','31037','31038','31042','31052','31053','31056','31061','31064','31066','31075','31086','31091','31094','31097','31098','31102','31112','31120','31121','31125','31133','31137','31158','31165','31174','31177','31183','31186','31194','31195','31196','31201','31204','31207','31213','31214','31218','31220','31221','31226','31229','31231','31235','31238','31245','31249','31252','31253','31259','31260','31269','31277','31279','31280','31285','31286','31288','31289','31295','31297','31298','31316','31327','31328','31336','31343','31349','31355','31359','31364','31365','31368','31371','31375','31381','31385','31387','31392','31393','31398','31400','31402','31406','31413','31431','31437','31441','31447','31453','31458','31463','31473','31493','31494','31496','31497','31498','31501','31504','31507','31510','31520','31521','31525','31526','31537','31544','31545','31547','31551','31554','31558','31561','31562','31568','31575','31579','31580','31582','31583','31585','31590','31608','31609','31620','31621','31622','31626','31628','31629','31630','31633','31640','31641','31643','31646','31650','31653','31663','31668','31684','31689','31696','31701','31704','31715','31728','31729','31731','31735','31739','31741','31742','31744','31745','31747','31754','31756','31762','31769','31770','31773','31774','31776','31777','31787','31790','31795','31803','31806','31817','31818','31824','31825','31826','31835','31840','31842','31844','31847','31850','31851','31860','31862','31871','31872','31888','31897','31911','31921','31923','31933','31935','31946','31949','31951','31956','31961','31975','31977','31983','31984','31987','31991','32001','32006','32014','32022','32025','32033','32037','32038','32045','32053','32060','32065','32067','32071','32072','32076','32077','32081','32082','32088','32092','32100','32102','32103','32109','32113','32118','32123','32128','32132','32134','32135','32138','32140','32142','32143','32146','32150','32151','32153','32157','32160','32161','32162','32166','32187','32191','32192','32193','32195','32202','32205','32206','32211','32213','32215','32217','32223','32238','32240','32242','32243','32247','32250','32252','32276','32280','32282','32283','32286','32295','32301','32304','32310','32311','32312','32323','32324','32326','32336','32343','32346','32348','32351','32356','32360','32361','32362','32365','32368','32372','32374','32376','32382','32392','32399','32402','32416','32417','32418','32421','32426','32438','32450','32455','32456','32457','32459','32461','32469','32470','32479','32480','32485','32489','32494','32501','32503','32511','32512','32518','32519','32520','32523','32525','32529','32536','32538','32542','32544','32551','32580','32585','32587','32592','32595','32596','32604','32616','32618','32622','32623','32624','32626','32629','32633','32636','32643','32647','32650','32654','32662','32683','32686','32690','32699','32701','32705','32707','32724','32728','32733','32735','32749','32772','32777','32786','32789','32795','32796','32800','32815','32818','32819','32823','32825','32829','32830','32834','32835','32840','32841','32855','32857','32865','32868','32869','32871','32873','32880','32883','32889','32891','32892','32893','32894','32900','32931','32939','32945','32946','32951','32958','32964','32974','32975','32976','32977','32980','32982','32985','33001','33002','33003','33008','33009','33012','33014','33015','33019','33027','33031','33033','33042','33044','33045','33049','33055','33069','33083','33089','33093','33094','33109','33115','33116','33124','33125','33126','33135','33144','33149','33165','33173','33185','33210','33229','33234','33235','33240','33264','33271','33278','33288','33289','33291','33295','33303','33304','33305','33333','33335','33336','33346','33355','33376','33384')

Which gave the following error:

An error occurred when executing the query.
The data provider said:
OGR[3] error 1: SQL Expression Parsing Error: memory exhausted. Occured around :
'30171','30175','30178','30185','30186','30191','30192','30193','30198','30201',
^

By splitting the IN clause in two parts with an "OR", by writing "id IN (...) OR "id IN (...), I managed to get my query working:

"id" IN ('0-E','1','6','14','23','25','38','49','51','61','81','85','90','93','114','138','143','144','148','158','159','168','176','182','185','195','212','220','222','223','227','233','235','246','248','255','261','262','293','295','301','323','328','330','338','341','347','352','361','365','366','371','372','375','382','387','400','404','406','413','421','424','426','428','433','434','436','444','455','471','474','478','485','486','513','529','532','534','538','548','557','560','562','565','571','584','587','589','593','602','604','609','619','631','649','653','668','672','673','681','685','686','694','703','704','706','707','711','716','731','734','741','752','757','765','777','782','786','791','796','799','805','807','814','821','823','831','836','839','849','867','871','884','888','890','892','894','896','914','915','928','935','936','955','961','966','971','972','976','977','1000','1014','1035','1038','1046','1047','1049','1097','1100','1101','1105','1110','1115','1117','1120','1121','1122','1130','1133','1160','1174','1177','1187','1195','1209','1223','1229','1232','1243','1260','1261','1262','1268','1279','1291','1299','1306','1307','1310','1312','1315','1318','1323','1326','1332','1340','1351','1354','1372','1381','1392','1393','1396','1413','1418','1428','1432','1438','1445','1446','1449','1453','1461','1466','1482','1489','1501','1502','1505','1513','1520','1528','1532','1537','1539','1545','1547','1556','1560','1561','1564','1567','1570','1578','1584','1587','1592','1594','1595','1597','1603','1614','1622','1643','1655','1659','1660','1667','1680','1685','1687','1688','1696','1697','1701','1737','1756','1757','1767','1777','1789','1790','1797','1814','1815','1826','1827','1836','1840','1842','1843','1844','1846','1853','1863','1864','1867','1878','1881','1883','1889','1894','1899','1903','1926','1927','1928','1936','1948','1957','1964','1980','2003','2005','2018','2025','2026','2033','2043','2044','2055','2057','2062','2072','2073','2074','2079','2088','2089','2092','2095','2101','2107','2119','2131','2153','2156','2157','2162','2171','2176','2187','2190','2196','2198','2209','2242','2256','2265','2285','2287','2288','2290','2299','2318','2319','2324','2340','2342','2363','2365','2373','2374','2377','2383','2387','2395','2402','2413','2414','2421','2433','2441','2453','2467','2478','2481','2483','2487','2499','2503','2514','2533','2544','2546','2550','2556','2565','2567','2573','2574','2577','2582','2588','2589','2598','2603','2610','2614','2615','2620','2632','2640','2663','2664','2684','2714','2722','2725','2726','2748','2750','2758','2761','2764','2775','2778','2786','2788','2790','2806','2814','2815','2817','2820','2829','2834','2837','2856','2875','2881','2898','2902','2908','2932','2948','2949','2955','2959','2969','2973','2979','3000','3001','3003','3009','3015','3018','3019','3038','3055','3057','3071','3081','3082','3092','3093','3094','3117','3119','3127','3132','3133','3140','3160','3165','3184','3188','3195','3200','3205','3215','3216','3226','3229','3230','3236','3255','3256','3259','3260','3287','3288','3297','3323','3325','3329','3331','3340','3345','3358','3365','3372','3376','3384','3388','3404','3405','3421','3423','3430','3450','3460','3468','3476','3477','3483','3487','3497','3499','3500','3508','3514','3515','3516','3527','3532','3533','3555','3576','3585','3602','3603','3631','3641','3643','3646','3648','3649','3651','3652','3658','3671','3673','3681','3687','3688','3693','3702','3713','3716','3719','3723','3762','3765','3769','3777','3786','3793','3802','3803','3807','3808','3814','3816','3821','3826','3829','3831','3837','3839','3841','3842','3845','3852','3855','3856','3860','3865','3869','3872','3877','3888','3889','3899','3903','3905','3916','3918','3926','3929','3941','3947','3950','3953','3962','3970','3982','4015','4018','4030','4031','4040','4041','4044','4050','4055','4069','4071','4072','4076','4084','4085','4087','4093','4095','4106','4109','4113','4118','4119','4121','4124','4125','4129','4153','4156','4165','4174','4180','4192','4208','4214','4220','4225','4228','4231','4240','4268','4277','4281','4294','4295','4297','4303','4318','4324','4325','4326','4327','4337','4343','4345','4351','4354','4359','4364','4365','4388','4392','4398','4404','4405','4424','4431','4445','4458','4466','4468','4470','4472','4480','4487','4492','4493','4498','4499','4504','4510','4513','4521','4531','4543','4546','4549','4552','4556','4557','4558','4561','4562','4576','4583','4586','4597','4614','4617','4627','4646','4649','4654','4664','4668','4714','4723','4728','4752','4760','4761','4763','4767','4772','4791','4814','4823','4825','4840','4841','4847','4850','4865','4873','4884','4893','4895','4905','4918','4920','4935','4936','4943','4953','4964','4969','4978','4979','4980','4994','5003','5015','5017','5019','5021','5028','5030','5033','5039','5041','5042','5046','5050','5052','5053','5062','5064','5072','5079','5083','5087','5098','5111','5113','5119','5121','5147','5148','5154','5157','5159','5178','5185','5186','5192','5193','5201','5223','5238','5240','5251','5253','5260','5280','5288','5296','5297','5305','5306','5310','5313','5316','5319','5327','5329','5346','5347','5348','5362','5370','5371','5372','5382','5400','5411','5416','5427','5433','5438','5440','5444','5447','5453','5458','5459','5461','5465','5472','5474','5475','5483','5487','5494','5504','5507','5527','5536','5539','5542','5548','5561','5563','5564','5569','5571','5581','5584','5587','5588','5604','5609','5610','5614','5635','5642','5652','5653','5662','5669','5673','5678','5683','5691','5693','5695','5699','5704','5730','5732','5735','5743','5748','5758','5765','5766','5768','5770','5774','5779','5792','5811','5814','5819','5829','5830','5831','5856','5862','5875','5876','5879','5883','5885','5899','5903','5907','5908','5913','5921','5934','5941','5948','5953','5960','5964','5969','5973','5981','5985','5992','5993','5996','5998','6002','6004','6008','6015','6018','6020','6028','6034','6036','6048','6049','6059','6067','6073','6081','6084','6086','6093','6114','6122','6130','6137','6138','6142','6143','6157','6163','6165','6167','6178','6187','6192','6195','6201','6205','6206','6209','6218','6230','6245','6254','6260','6264','6275','6280','6281','6284','6305','6308','6312','6325','6326','6334','6359','6360','6362','6367','6373','6379','6389','6390','6394','6403','6417','6420','6423','6438','6443','6451','6459','6463','6466','6468','6470','6477','6482','6491','6500','6505','6509','6514','6518','6520','6525','6528','6541','6544','6547','6551','6559','6574','6585','6589','6594','6606','6609','6619','6627','6661','6668','6670','6671','6674','6680','6688','6696','6699','6707','6713','6718','6724','6745','6751','6755','6773','6781','6798','6810','6812','6844','6859','6860','6877','6878','6879','6888','6889','6891','6893','6901','6909','6912','6925','6926','6932','6935','6942','6943','6950','6957','6970','6976','6981','6987','6989','6990','6991','6997','7001','7005','7026','7027','7035','7048','7053','7057','7059','7061','7064','7080','7083','7089','7090','7091','7092','7094','7096','7104','7106','7108','7111','7112','7115','7129','7137','7144','7155','7157','7167','7177','7180','7200','7202','7214','7232','7235','7238','7248','7252','7256','7268','7272','7287','7292','7301','7322','7324','7326','7331','7333','7346','7349','7356','7359','7361','7371','7376','7377','7387','7388','7394','7401','7426','7427','7432','7435','7437','7438','7445','7446','7451','7454','7468','7485','7486','7514','7517','7536','7540','7544','7551','7555','7557','7572','7577','7589','7604','7606','7626','7627','7633','7642','7643','7652','7654','7668','7690','7696','7697','7702','7717','7721','7743','7746','7748','7749','7750','7751','7760','7764','7765','7772','7777','7779','7808','7813','7828','7863','7870','7876','7884','7889','7897','7898','7900','7903','7911','7912','7919','7922','7925','7934','7936','7944','7945','7947','7962','7964','7969','7975','7978','7990','7994','8000','8015','8023','8028','8035','8040','8041','8048','8051','8055','8056','8057','8062','8071','8080','8083','8086','8097','8102','8106','8113','8120','8135','8139','8152','8164','8172','8174','8175','8178','8182','8191','8199','8205','8210','8211','8216','8221','8223','8224','8232','8240','8241','8252','8275','8292','8303','8306','8307','8321','8326','8335','8350','8358','8365','8374','8376','8386','8387','8396','8398','8404','8411','8433','8443','8456','8458','8479','8482','8489','8496','8499','8504','8511','8521','8523','8524','8529','8530','8534','8538','8548','8549','8555','8558','8561','8562','8571','8579','8589','8593','8606','8614','8628','8629','8638','8640','8653','8661','8669','8676','8694','8698','8720','8723','8729','8733','8742','8745','8752','8756','8758','8763','8774','8791','8795','8809','8816','8820','8821','8836','8837','8838','8840','8845','8849','8851','8853','8856','8865','8867','8877','8878','8892','8901','8906','8912','8914','8915','8923','8941','8946','8956','8972','8981','8986','8991','8992','9007','9023','9026','9039','9041','9049','9074','9077','9086','9096','9101','9106','9114','9120','9132','9135','9137','9138','9159','9161','9164','9183','9189','9204','9210','9217','9220','9226','9228','9231','9236','9237','9239','9243','9245','9248','9250','9260','9262','9264','9274','9277','9299','9310','9311','9314','9316','9336','9356','9365','9367','9368','9370','9374','9378','9393','9398','9411','9415','9439','9440','9452','9468','9470','9474','9476','9477','9484','9493','9507','9516','9517','9534','9538','9542','9554','9557','9569','9570','9579','9587','9602','9613','9626','9636','9672','9691','9713','9725','9739','9744','9747','9754','9758','9760','9763','9765','9772','9775','9778','9779','9792','9801','9818','9826','9864','9871','9875','9876','9920','9921','9923','9929','9939','9943','9949','9953','9957','9971','9984','9991','10007','10010','10017','10020','10026','10027','10034','10046','10053','10055','10058','10060','10061','10063','10067','10077','10078','10095','10101','10106','10132','10133','10135','10140','10149','10150','10157','10160','10163','10181','10190','10192','10208','10221','10226','10243','10251','10256','10271','10274','10279','10289','10296','10305','10314','10348','10368','10372','10373','10379','10388','10403','10417','10428','10429','10432','10433','10435','10436','10442','10444','10449','10452','10453','10456','10460','10472','10495','10504','10513','10540','10546','10551','10561','10580','10586','10587','10589','10599','10612','10632','10642','10644','10645','10647','10660','10679','10681','10682','10684','10685','10687','10692','10693','10697','10709','10715','10720','10738','10745','10748','10751','10760','10762','10773','10788','10797','10798','10800','10819','10820','10826','10830','10859','10860','10863','10867','10877','10884','10895','10896','10902','10909','10910','10911','10912','10914','10916','10918','10921','10924','10937','10941','10968','10980','10981','10984','10987','10988','10993','10997','11003','11023','11026','11051','11056','11066','11077','11082','11086','11091','11092','11102','11108','11110','11118','11128','11138','11141','11146','11174','11177','11180','11194','11195','11205','11208','11213','11217','11220','11224','11243','11244','11253','11255','11262','11270','11272','11285','11287','11296','11297','11301','11307','11308','11318','11321','11339','11346','11356','11364','11366','11370','11375','11377','11380','11404','11405','11410','11414','11417','11446','11448','11451','11459','11464','11476','11479','11502','11512','11515','11537','11541','11542','11544','11551','11563','11565','11578','11581','11585','11592','11595','11599','11600','11605','11618','11630','11637','11643','11648','11652','11654','11657','11664','11667','11671','11674','11677','11678','11688','11698','11699','11701','11705','11707','11710','11721','11724','11728','11736','11751','11757','11758','11762','11771','11780','11783','11784','11796','11798','11801','11809','11813','11821','11826','11844','11845','11848','11854','11858','11863','11864','11867','11871','11874','11891','11892','11894','11897','11902','11907','11910','11912','11914','11917','11943','11947','11950','11958','11967','11999','12023','12028','12036','12039','12060','12066','12074','12076','12080','12082','12086','12087','12096','12106','12115','12120','12126','12136','12139','12143','12150','12155','12159','12164','12182','12193','12195','12196','12205','12213','12238','12242','12244','12253','12257','12271','12281','12284','12292','12295','12301','12318','12329','12337','12338','12352','12360','12363','12364','12374','12382','12401','12432','12438','12440','12443','12444','12455','12459','12461','12464','12486','12491','12498','12510','12513','12514','12534','12537','12539','12540','12551','12564','12567','12573','12574','12581','12584','12586','12600','12603','12611','12615','12625','12629','12638','12642','12656','12659','12661','12671','12677','12678','12684','12692','12693','12695','12697','12716','12724','12735','12756','12775','12787','12790','12805','12809','12824','12835','12837','12844','12845','12857','12861','12868','12876','12880','12881','12884','12893','12903','12908','12923','12926','12935','12940','12947','12953','12956','12965','12977','12979','12980','12984','12992','12996','12998','13002','13012','13027','13062','13066','13081','13087','13100','13103','13106','13107','13117','13120','13123','13131','13146','13148','13149','13154','13160','13162','13166','13170','13184','13185','13186','13190','13192','13193','13196','13215','13216','13218','13219','13230','13238','13239','13244','13247','13253','13259','13265','13266','13269','13282','13285','13286','13289','13300','13301','13307','13312','13313','13318','13320','13331','13332','13357','13358','13366','13367','13375','13389','13407','13408','13409','13410','13416','13429','13433','13438','13440','13459','13465','13470','13475','13481','13482','13483','13486','13487','13490','13502','13505','13506','13510','13514','13527','13540','13545','13546','13547','13564','13565','13566','13570','13572','13576','13578','13580','13581','13583','13585','13591','13593','13595','13597','13601','13606','13611','13619','13624','13631','13635','13637','13638','13639','13651','13660','13678','13683','13686','13726','13729','13732','13737','13738','13741','13742','13756','13759','13762','13765','13769','13770','13775','13784','13792','13805','13808','13820','13827','13834','13841','13855','13857','13862','13868','13871','13881','13882','13889','13891','13896','13905','13911','13913','13926','13929','13932','13942','13962','13989','13990','13994','13995','13997','14003','14004','14006','14007','14010','14013','14015','14017','14019','14024','14026','14030','14033','14034','14069','14075','14078','14080','14081','14082','14085','14089','14090','14091','14093','14097','14099','14106','14107','14113','14116','14135','14147','14149','14153','14158','14160','14168','14169','14175','14176','14190','14192','14204','14206','14207','14218','14223','14231','14248','14249','14252','14258','14261','14266','14267','14272','14274','14280','14281','14286','14295','14297','14308','14319','14322','14324','14327','14328','14330','14332','14336','14337','14341','14351','14363','14369','14373','14376','14377','14378','14380','14381','14386','14390','14393','14398','14410','14417','14436','14438','14445','14447','14450','14451','14454','14457','14466','14467','14469','14472','14474','14480','14481','14485','14486','14495','14504','14509','14523','14526','14538','14555','14577','14595','14596','14602','14610','14611','14641','14646','14651','14652','14657','14688','14700','14707','14709','14722','14726','14727','14730','14735','14744','14750','14761','14766','14770','14778','14789','14795','14803','14804','14813','14820','14826','14837','14842','14849','14854','14856','14862','14874','14878','14879','14880','14909','14925','14931','14936','14945','14961','14966','14968','14975','14989','14991','14996','14999','15013','15024','15025','15047','15049','15060','15061','15064','15074','15088','15092','15105','15111','15112','15118','15119','15121','15123','15128','15135','15143','15155','15156','15164','15168','15172','15173','15175','15177','15181','15192','15196','15201','15203','15212','15213','15214','15216','15224','15229','15234','15235','15236','15250','15253','15255','15261','15262','15267','15274','15277','15287','15320','15321','15329','15331','15332','15338','15340','15341','15352','15356','15359','15369','15379','15381','15386','15390','15394','15399','15406','15412','15413','15423','15428','15433','15442','15448','15455','15456','15460','15472','15473','15491','15494','15512','15529','15538','15559','15570','15582','15586','15588','15591','15592','15599','15603','15605','15606','15610','15613','15631','15632','15656','15664','15665','15670','15671','15677','15685','15697','15703','15719','15725','15731','15736','15739','15745','15746','15752','15753','15759','15770','15773','15775','15778','15782','15785','15786','15792','15796','15801','15804','15809','15810','15815','15817','15821','15827','15832','15833','15849','15855','15858','15866','15874','15879','15884','15894','15896','15907','15908','15915','15916','15930','15943','15952','15955','15960','15962','15973','15976','15989','15995','16002','16003','16006','16007','16017','16023','16029','16047','16049','16058','16065','16075','16092','16101','16107','16108','16123','16138','16141','16144','16155','16159','16171','16188','16192','16200','16201','16205','16208','16209','16214','16215','16218','16220','16221','16223','16224','16232','16242','16243','16252','16255','16261','16266','16267','16270','16271','16272','16276','16277','16288','16290','16299','16303','16305','16306','16315','16336','16342','16349','16355','16362','16371','16377','16378','16386','16390','16391','16434','16442','16448','16452','16462','16463','16466','16468','16491','16500','16502','16504','16519','16524','16536','16539','16542','16549','16555','16561','16565','16570','16576','16579','16594','16599','16603','16604','16605','16622','16623','16629','16635','16639','16640','16649','16657','16659','16662','16663','16666','16670','16671','16679','16687','16694','16696','16698','16700','16715','16723','16731','16733','16739','16743','16754','16755','16758','16762','16777','16800','16803','16806','16809','16828','16845','16867','16870','16882','16883','16891','16906','16909','16912','16913','16919','16922','16925','16926','16945','16947','16949','16967','16978','16981','16987','16990','16998','17001','17011','17012','17022','17024','17025','17028','17038','17040','17041','17042','17048','17050','17056','17061','17067','17070','17073','17093','17094','17105','17107','17110','17112','17114','17118','17127','17135','17149','17156','17165','17168','17171','17175','17187','17188','17193','17194','17197','17207','17209','17211','17218','17219','17221','17222','17226','17229','17240','17247','17251','17253','17276','17283','17292','17295','17302','17304','17315','17324','17326','17329','17331','17336','17347','17351','17353','17362','17363','17370','17377','17386','17388','17390','17399','17403','17404','17405','17407','17408','17411','17419','17435','17437','17441','17451','17463','17475','17482','17484','17485','17491','17493','17496','17503','17505','17507','17509','17520','17521','17529','17534','17535','17547','17558','17560','17568','17570','17579','17584','17587','17596','17598','17600','17601','17602','17605','17616','17632','17639','17648','17649','17652','17656','17660','17680','17681','17684','17691','17700','17701','17706','17710','17724','17729','17741','17742','17747','17750','17757','17762','17774','17775','17782','17786','17790','17804','17813','17830','17831','17832','17833','17835','17837','17842','17843','17845','17846','17847','17851','17854','17858','17861','17863','17866','17887','17897','17914','17919','17921','17922','17933','17941','17950','17961','17968','17974','17983','17988','17989','17998','17999','18000','18006','18019','18025','18035','18036','18049','18050','18058','18069','18077','18078','18092','18098','18103','18105','18114','18118','18119','18125','18127','18136','18138','18144','18149','18153','18158','18163','18166','18176','18181','18184','18192','18196','18209','18228','18229','18235','18236','18246','18269','18272','18283','18288','18290','18293','18295','18312','18326','18332','18335','18337','18343','18345','18353','18358','18364','18380','18382','18396','18402','18403','18417','18423','18427','18432','18435','18441','18454','18457','18461','18481','18488','18489','18494','18495','18502','18506','18518','18535','18536','18549','18554','18559','18595','18596','18612','18623','18625','18628','18629','18634','18637','18642','18645','18656','18662','18664','18673','18687','18693','18702','18715','18716','18719','18720','18725','18730','18742','18743','18746','18748','18755','18757','18772','18775','18776','18781','18787','18788','18790','18797','18809','18811','18813','18816','18823','18825','18851','18858','18871','18872','18875','18891','18903','18906','18907','18908','18910','18913','18915','18920','18937','18941','18942','18943','18946','18948','18949','18954','18957','18958','18962','18964','18968','18976','18981','18988','18990','18996','19005','19014','19019','19023','19031','19037','19040','19042','19058','19068','19070','19074','19079','19086','19091','19094','19095','19105','19110','19112','19115','19121','19122','19123','19136','19145','19157','19165','19169','19171','19176','19181','19186','19189','19194','19197','19199','19207','19208','19214','19223','19225','19228','19237','19249','19252','19255','19264','19268','19269','19270','19274','19295','19301','19307','19308','19314','19317','19318','19321','19323','19324','19335','19337','19339','19343','19348','19349','19353','19358','19364','19371','19382','19384','19385','19395','19398','19402','19405','19422','19430','19447','19448','19453','19454','19456','19465','19466','19476','19478','19484','19486','19488','19490','19501','19506','19514','19519','19521','19522','19527','19539','19540','19553','19555','19557','19564','19577','19579','19580','19590','19604','19612','19614','19623','19631','19634','19642','19648','19651','19654','19655','19656','19668','19677','19682','19686','19695','19721','19724','19730','19749','19751','19753','19755','19759','19762','19766','19767','19768','19772','19778','19791','19807','19812','19818','19829','19836','19838','19843','19847','19848','19850','19852','19853','19854','19858','19861','19868','19870','19878','19884','19910','19914','19918','19921','19922','19925','19928','19942','19943','19947','19950','19959','19971','19977','19981','19985','19986','19994','20000','20003','20007','20008','20015','20028','20029','20041','20056','20063','20069','20075','20078','20083','20085','20089','20091','20098','20105','20128','20129','20143','20146','20153','20154','20156','20159','20170','20180','20181','20182','20183','20188','20189','20195','20200','20204','20209','20216','20218','20221','20224','20229','20233','20234','20237','20239','20241','20253','20256','20260','20263','20264','20265','20268','20271','20274','20289','20290','20291','20304','20306','20307','20308','20311','20317','20324','20339','20348','20350','20357','20363','20369','20370','20376','20379','20387','20388','20403','20404','20407','20413','20415','20425','20433','20437','20440','20442','20443','20444','20446','20448','20450','20458','20464','20481','20489','20495','20496','20511','20512','20513','20514','20517','20522','20534','20535','20540','20541','20550','20555','20556','20558','20561','20569','20570','20573','20582','20613','20614','20623','20631','20637','20643','20647','20669','20686','20695','20696','20705','20707','20716','20718','20724','20728','20729','20736','20738','20742','20748','20761','20779','20782','20785','20793','20812','20814','20840','20846','20848','20853','20857','20872','20875','20880','20881','20885','20891','20899','20914','20917','20924','20937','20943','20944','20947','20959','20970','20974','20982','20984','20985','20986','21003','21006','21013','21028','21032','21036','21047','21054','21056','21057','21058','21079','21087','21094','21097','21102','21111','21113','21118','21120','21131','21138','21153','21154','21167','21168','21170','21172','21186','21187','21191','21202','21213','21216','21217','21220','21227','21238','21241','21247','21249','21271','21281','21290','21300','21304','21306','21312','21316','21318','21320','21328','21334','21337','21338','21348','21350','21352','21356','21399','21403','21423','21424','21435','21441','21449','21455','21463','21478','21481','21482','21486','21487','21491','21494','21509','21510','21512','21518','21521','21523','21529','21530','21532','21533','21535','21537','21549','21577','21578','21579','21583','21587','21598','21604','21613','21623','21632','21641','21643','21652','21655','21656','21659','21671','21673','21674','21683','21684','21690','21697','21713','21716','21721','21725','21727','21733','21734','21735','21737','21745','21746','21751','21753','21758','21775','21779','21788','21789','21793','21795','21800','21809','21813','21814','21826','21831','21850','21858','21862','21869','21873','21875')
OR "id" IN ('21877','21880','21886','21892','21893','21896','21898','21900','21903','21917','21919','21923','21926','21934','21936','21938','21940','21950','21952','21961','21968','21972','21978','21985','21994','22002','22008','22009','22011','22014','22015','22017','22019','22026','22032','22045','22065','22067','22079','22088','22096','22105','22107','22109','22111','22116','22123','22136','22146','22150','22161','22176','22184','22196','22201','22209','22212','22215','22217','22223','22224','22229','22232','22234','22235','22236','22249','22252','22255','22259','22262','22268','22269','22272','22276','22278','22292','22308','22310','22312','22315','22316','22326','22338','22340','22341','22342','22347','22351','22368','22374','22386','22389','22393','22397','22399','22400','22412','22414','22421','22426','22441','22442','22443','22446','22449','22465','22467','22490','22492','22494','22497','22499','22506','22508','22512','22516','22519','22522','22527','22548','22552','22554','22559','22569','22574','22575','22576','22581','22583','22586','22593','22604','22609','22610','22617','22623','22625','22633','22634','22637','22638','22640','22642','22645','22647','22651','22661','22662','22665','22666','22677','22678','22681','22682','22683','22685','22694','22695','22701','22702','22708','22713','22724','22726','22729','22731','22741','22745','22747','22748','22761','22782','22784','22795','22796','22798','22800','22804','22823','22833','22835','22844','22847','22850','22854','22857','22858','22864','22872','22875','22883','22891','22902','22904','22911','22934','22954','22970','22972','22974','22983','22993','23007','23009','23012','23013','23017','23023','23037','23046','23047','23048','23049','23061','23072','23083','23091','23109','23113','23114','23115','23116','23125','23127','23128','23133','23140','23143','23145','23147','23154','23160','23173','23180','23182','23186','23187','23204','23207','23211','23215','23217','23223','23224','23239','23257','23266','23268','23275','23277','23282','23287','23288','23289','23290','23301','23302','23306','23314','23315','23319','23322','23331','23336','23341','23344','23345','23348','23350','23354','23355','23365','23368','23369','23371','23372','23375','23381','23387','23394','23395','23398','23408','23415','23435','23438','23448','23453','23454','23462','23465','23466','23478','23481','23486','23490','23491','23498','23516','23519','23528','23536','23537','23541','23544','23549','23550','23553','23554','23555','23557','23560','23568','23569','23570','23578','23587','23588','23595','23597','23602','23605','23613','23615','23619','23620','23626','23627','23631','23638','23648','23654','23655','23658','23665','23668','23680','23700','23707','23711','23718','23722','23723','23725','23727','23731','23734','23740','23747','23759','23764','23766','23784','23786','23790','23794','23802','23811','23814','23819','23821','23827','23830','23834','23843','23846','23861','23863','23865','23867','23870','23875','23896','23898','23899','23910','23911','23912','23920','23938','23939','23946','23949','23971','23972','23975','23977','23981','23984','23987','23997','24002','24003','24009','24012','24023','24024','24031','24034','24043','24048','24051','24056','24062','24070','24074','24077','24079','24091','24094','24095','24109','24110','24113','24116','24122','24125','24128','24129','24136','24141','24148','24150','24152','24160','24161','24162','24163','24166','24170','24171','24172','24175','24179','24181','24183','24196','24210','24211','24225','24232','24241','24244','24245','24249','24250','24258','24262','24267','24270','24272','24274','24279','24295','24303','24304','24313','24320','24321','24326','24328','24332','24335','24340','24347','24349','24354','24355','24356','24358','24362','24365','24367','24373','24374','24377','24386','24389','24405','24407','24426','24430','24432','24441','24445','24447','24448','24473','24479','24485','24486','24490','24495','24496','24506','24512','24514','24516','24526','24536','24540','24544','24546','24565','24569','24576','24578','24582','24583','24584','24585','24587','24592','24595','24599','24600','24603','24608','24618','24624','24625','24626','24633','24639','24641','24645','24649','24665','24668','24673','24678','24679','24687','24689','24695','24696','24699','24705','24708','24711','24714','24715','24716','24719','24720','24721','24747','24751','24755','24780','24784','24787','24789','24792','24797','24817','24824','24825','24826','24827','24828','24837','24845','24858','24869','24875','24892','24896','24899','24907','24908','24909','24910','24911','24913','24914','24916','24921','24933','24935','24953','24954','24955','24958','24965','24968','24994','25002','25012','25014','25016','25021','25028','25032','25051','25054','25056','25057','25061','25076','25085','25089','25091','25092','25097','25098','25107','25112','25114','25116','25117','25125','25128','25133','25137','25142','25144','25146','25148','25159','25160','25166','25167','25170','25180','25181','25185','25186','25197','25198','25203','25204','25211','25227','25229','25232','25235','25239','25242','25244','25245','25259','25261','25263','25268','25272','25282','25284','25289','25290','25291','25294','25297','25300','25303','25306','25322','25323','25326','25332','25333','25337','25339','25349','25351','25352','25354','25358','25360','25362','25394','25395','25398','25401','25402','25403','25408','25409','25416','25429','25435','25450','25451','25452','25458','25459','25486','25491','25495','25496','25503','25505','25508','25513','25514','25519','25520','25522','25524','25526','25528','25530','25534','25535','25544','25556','25558','25560','25563','25571','25572','25578','25581','25586','25595','25598','25599','25600','25601','25617','25622','25631','25638','25640','25641','25644','25648','25649','25650','25652','25654','25660','25669','25673','25676','25699','25700','25701','25705','25708','25721','25723','25730','25735','25739','25745','25755','25765','25770','25773','25777','25782','25797','25800','25803','25820','25825','25829','25830','25835','25848','25856','25858','25859','25861','25869','25875','25881','25885','25890','25891','25896','25897','25899','25911','25913','25921','25922','25923','25925','25926','25932','25934','25940','25942','25946','25954','25959','25960','25971','25983','25985','26012','26015','26018','26024','26034','26041','26049','26053','26055','26058','26062','26074','26079','26090','26098','26118','26123','26141','26154','26155','26156','26159','26163','26166','26182','26184','26185','26187','26190','26198','26205','26212','26217','26228','26229','26230','26232','26237','26239','26243','26252','26253','26254','26255','26257','26263','26264','26265','26272','26273','26284','26291','26297','26324','26335','26338','26346','26352','26361','26371','26375','26376','26379','26380','26384','26391','26394','26395','26403','26404','26412','26420','26425','26432','26434','26436','26438','26444','26447','26449','26472','26478','26481','26497','26498','26500','26501','26508','26513','26514','26521','26523','26524','26525','26533','26536','26549','26555','26558','26560','26583','26585','26586','26598','26613','26617','26623','26633','26649','26652','26675','26676','26678','26680','26686','26692','26694','26696','26700','26703','26706','26710','26712','26716','26717','26718','26720','26721','26724','26728','26730','26733','26746','26754','26758','26759','26766','26767','26769','26781','26790','26797','26802','26815','26818','26824','26830','26837','26842','26856','26858','26864','26872','26874','26875','26876','26882','26883','26885','26887','26889','26894','26895','26930','26937','26939','26945','26960','26976','26979','26980','26983','26985','26990','27002','27011','27012','27014','27034','27036','27037','27046','27048','27053','27058','27064','27088','27094','27106','27111','27118','27121','27122','27127','27131','27133','27137','27142','27144','27145','27146','27148','27150','27152','27153','27154','27158','27165','27166','27172','27173','27175','27179','27189','27194','27197','27203','27206','27225','27229','27242','27246','27256','27272','27274','27286','27288','27297','27303','27313','27316','27317','27319','27322','27323','27324','27325','27327','27330','27334','27354','27359','27365','27367','27381','27382','27383','27388','27395','27396','27400','27407','27408','27411','27415','27422','27428','27429','27437','27442','27447','27453','27455','27457','27467','27470','27479','27487','27489','27490','27499','27506','27511','27513','27516','27517','27521','27525','27534','27539','27546','27549','27554','27555','27558','27560','27566','27573','27574','27582','27589','27593','27595','27601','27607','27612','27623','27626','27630','27631','27637','27638','27641','27642','27644','27645','27646','27653','27655','27662','27663','27681','27684','27693','27696','27698','27707','27708','27711','27715','27719','27722','27724','27728','27729','27739','27750','27756','27768','27769','27773','27777','27785','27791','27800','27803','27804','27808','27809','27810','27813','27819','27821','27824','27830','27844','27845','27846','27853','27857','27870','27872','27882','27888','27891','27900','27907','27908','27914','27915','27917','27925','27927','27930','27932','27933','27935','27936','27937','27939','27940','27941','27943','27949','27955','27960','27962','27966','27969','27976','27978','27986','28009','28013','28017','28026','28028','28030','28032','28033','28036','28041','28044','28045','28047','28051','28058','28063','28064','28066','28068','28069','28070','28073','28083','28085','28090','28094','28097','28100','28109','28111','28118','28119','28120','28132','28138','28142','28143','28147','28158','28162','28164','28169','28174','28175','28181','28186','28189','28190','28192','28195','28201','28214','28224','28226','28227','28231','28245','28251','28258','28262','28270','28274','28284','28290','28292','28298','28307','28323','28327','28330','28337','28339','28340','28346','28375','28376','28384','28386','28389','28394','28395','28398','28414','28415','28421','28424','28434','28444','28455','28464','28473','28476','28479','28481','28483','28484','28485','28490','28491','28494','28508','28509','28523','28524','28526','28545','28548','28549','28556','28558','28568','28572','28578','28579','28582','28590','28591','28596','28600','28603','28613','28616','28618','28620','28623','28634','28635','28640','28641','28643','28651','28659','28660','28661','28665','28668','28669','28680','28686','28693','28694','28695','28700','28701','28703','28704','28711','28712','28720','28724','28736','28740','28741','28742','28746','28747','28751','28756','28758','28767','28769','28773','28775','28777','28778','28791','28808','28817','28819','28822','28823','28831','28835','28841','28845','28847','28850','28852','28856','28862','28865','28867','28869','28872','28880','28897','28899','28901','28907','28908','28913','28914','28915','28923','28926','28934','28939','28940','28945','28947','28948','28949','28964','28967','28972','28987','28990','29002','29005','29012','29018','29026','29033','29035','29038','29039','29040','29054','29058','29065','29069','29070','29079','29083','29090','29095','29108','29109','29112','29115','29120','29125','29127','29136','29139','29141','29143','29145','29146','29157','29162','29163','29178','29190','29193','29200','29207','29208','29212','29222','29223','29227','29229','29235','29236','29238','29242','29250','29252','29260','29266','29271','29273','29281','29282','29289','29295','29296','29301','29305','29306','29308','29310','29314','29316','29323','29329','29332','29335','29336','29338','29344','29346','29347','29351','29352','29356','29360','29364','29369','29370','29375','29376','29382','29391','29394','29406','29407','29410','29423','29437','29441','29444','29446','29449','29453','29454','29468','29469','29471','29474','29477','29479','29480','29481','29487','29504','29510','29516','29522','29527','29537','29538','29539','29543','29552','29554','29555','29556','29562','29569','29571','29575','29576','29579','29587','29592','29594','29595','29615','29622','29623','29624','29625','29630','29631','29632','29635','29640','29642','29647','29648','29650','29659','29663','29671','29675','29684','29693','29694','29700','29701','29706','29722','29729','29731','29732','29735','29740','29751','29757','29759','29761','29763','29769','29770','29771','29779','29780','29783','29785','29787','29788','29793','29794','29795','29797','29808','29811','29813','29831','29833','29839','29842','29847','29848','29851','29866','29867','29870','29877','29881','29883','29884','29893','29898','29905','29908','29911','29914','29916','29918','29920','29926','29939','29953','29956','29965','29978','29985','29992','29997','30009','30011','30017','30026','30027','30029','30031','30034','30040','30044','30045','30047','30048','30054','30063','30064','30068','30071','30074','30077','30079','30081','30096','30098','30101','30104','30111','30116','30129','30131','30141','30147','30156','30160','30162','30169','30171','30175','30178','30185','30186','30191','30192','30193','30198','30201','30203','30204','30209','30220','30222','30233','30239','30246','30248','30249','30255','30257','30258','30260','30269','30271','30274','30279','30280','30281','30283','30287','30288','30296','30300','30308','30309','30324','30328','30331','30333','30335','30339','30341','30343','30344','30346','30348','30352','30356','30358','30362','30372','30375','30378','30379','30388','30394','30400','30404','30422','30435','30440','30445','30447','30449','30458','30462','30474','30480','30488','30489','30500','30501','30503','30504','30506','30507','30509','30516','30521','30524','30528','30529','30538','30540','30548','30549','30558','30560','30562','30575','30583','30587','30591','30595','30596','30598','30605','30606','30609','30610','30611','30613','30618','30627','30628','30629','30635','30641','30645','30646','30659','30660','30669','30670','30680','30682','30683','30690','30692','30696','30697','30701','30703','30704','30708','30715','30717','30733','30740','30749','30751','30754','30760','30762','30765','30766','30768','30770','30772','30777','30784','30787','30792','30799','30807','30808','30811','30815','30817','30818','30822','30833','30836','30837','30852','30853','30854','30864','30870','30873','30881','30883','30885','30888','30890','30896','30904','30905','30908','30909','30916','30920','30925','30926','30927','30935','30936','30943','30955','30956','30962','30971','30974','30975','30979','30981','30982','30983','30988','30989','31006','31010','31022','31025','31032','31037','31038','31042','31052','31053','31056','31061','31064','31066','31075','31086','31091','31094','31097','31098','31102','31112','31120','31121','31125','31133','31137','31158','31165','31174','31177','31183','31186','31194','31195','31196','31201','31204','31207','31213','31214','31218','31220','31221','31226','31229','31231','31235','31238','31245','31249','31252','31253','31259','31260','31269','31277','31279','31280','31285','31286','31288','31289','31295','31297','31298','31316','31327','31328','31336','31343','31349','31355','31359','31364','31365','31368','31371','31375','31381','31385','31387','31392','31393','31398','31400','31402','31406','31413','31431','31437','31441','31447','31453','31458','31463','31473','31493','31494','31496','31497','31498','31501','31504','31507','31510','31520','31521','31525','31526','31537','31544','31545','31547','31551','31554','31558','31561','31562','31568','31575','31579','31580','31582','31583','31585','31590','31608','31609','31620','31621','31622','31626','31628','31629','31630','31633','31640','31641','31643','31646','31650','31653','31663','31668','31684','31689','31696','31701','31704','31715','31728','31729','31731','31735','31739','31741','31742','31744','31745','31747','31754','31756','31762','31769','31770','31773','31774','31776','31777','31787','31790','31795','31803','31806','31817','31818','31824','31825','31826','31835','31840','31842','31844','31847','31850','31851','31860','31862','31871','31872','31888','31897','31911','31921','31923','31933','31935','31946','31949','31951','31956','31961','31975','31977','31983','31984','31987','31991','32001','32006','32014','32022','32025','32033','32037','32038','32045','32053','32060','32065','32067','32071','32072','32076','32077','32081','32082','32088','32092','32100','32102','32103','32109','32113','32118','32123','32128','32132','32134','32135','32138','32140','32142','32143','32146','32150','32151','32153','32157','32160','32161','32162','32166','32187','32191','32192','32193','32195','32202','32205','32206','32211','32213','32215','32217','32223','32238','32240','32242','32243','32247','32250','32252','32276','32280','32282','32283','32286','32295','32301','32304','32310','32311','32312','32323','32324','32326','32336','32343','32346','32348','32351','32356','32360','32361','32362','32365','32368','32372','32374','32376','32382','32392','32399','32402','32416','32417','32418','32421','32426','32438','32450','32455','32456','32457','32459','32461','32469','32470','32479','32480','32485','32489','32494','32501','32503','32511','32512','32518','32519','32520','32523','32525','32529','32536','32538','32542','32544','32551','32580','32585','32587','32592','32595','32596','32604','32616','32618','32622','32623','32624','32626','32629','32633','32636','32643','32647','32650','32654','32662','32683','32686','32690','32699','32701','32705','32707','32724','32728','32733','32735','32749','32772','32777','32786','32789','32795','32796','32800','32815','32818','32819','32823','32825','32829','32830','32834','32835','32840','32841','32855','32857','32865','32868','32869','32871','32873','32880','32883','32889','32891','32892','32893','32894','32900','32931','32939','32945','32946','32951','32958','32964','32974','32975','32976','32977','32980','32982','32985','33001','33002','33003','33008','33009','33012','33014','33015','33019','33027','33031','33033','33042','33044','33045','33049','33055','33069','33083','33089','33093','33094','33109','33115','33116','33124','33125','33126','33135','33144','33149','33165','33173','33185','33210','33229','33234','33235','33240','33264','33271','33278','33288','33289','33291','33295','33303','33304','33305','33333','33335','33336','33346','33355','33376','33384')

And this is the result :-)

dismo pwdSample and ssb for a large number of species distribution records

2016-03-21T17:17:00.000+01:00

For my MarineSPEED dataset of species I'm trying to create non-random cross-validation datasets. In order to do this I was looking at the pwdSample and ssb functions from the dismo package. pwdSample calculates a pair-wise distance sampling and ssb calculates the spatial sorting bias. More information about these functions can be found in the excellent paper by Robert Hijmans (2012): "Cross-validation of species distribution models: removing spatial sorting bias and calibration with a null model".

Problem is that both functions calculate a distance matrix between two sets of points which starts to fail when the number of points is (very) large. To fix this I re-implemented the functions replacing the matrix calculation and subsequent row minimum finding with the same functionality but using a divide and conquer strategy by calculating distance matrices on subsets, taking the min distance and then taking the min distances of these min distances to get row based minima.

In other words I replaced:

fromd <- apply(distfun(fixed, reference), 1, min)

with:

mindist <- function(distfun, a, b) {
  partition_count <- (1 %/% (100 / NROW(b))) + 1
  parts <- dismo::kfold(x=b, k=partition_count)
  r <- c()
  for(i in 1:partition_count) {
    mind <- apply(distfun(a, b[parts==i,]), 1, min)
    r <- cbind(r, mind)
  }
  apply(r, 1, min)
}
  
fromd <- mindist(distfun, fixed, reference)

The full pwdSample function then becomes:

pwdSample_robust <- function(fixed, sample, reference, tr = 0.33, nearest= TRUE, n=1, lonlat = TRUE, warn = TRUE) {
  ## DIVIDE reference, calculate distance matrix, get min, COMBINE: take min of the min, do rest of the pwdSample logic
  distHaversine <- function(p1, p2) {
    r <- 6378137
    toRad <- pi/180
    p1 <- p1 * toRad
    p2 <- p2 * toRad
    p <- cbind(p1[, 1], p1[, 2], p2[, 1], p2[, 2])
    dLat <- (p[, 4] - p[, 2])
    dLon <- (p[, 3] - p[, 1])
    a <- sin(dLat/2) * sin(dLat/2) + cos(p[, 2]) * cos(p[, 
                                                         4]) * sin(dLon/2) * sin(dLon/2)
    dist <- 2 * atan2(sqrt(a), sqrt(1 - a)) * r
    as.vector(dist)
  }
  distGeo <- function(x, y) {
    n <- nrow(x)
    m <- nrow(y)
    dm <- matrix(ncol = m, nrow = n)
    for (i in 1:n) {
      dm[i, ] <- distHaversine(x[i, , drop = FALSE], y)
    }
    return(dm)
  }
  distPlane <- function(x, y) {
    dfun <- function(x, y) {
      sqrt((x[, 1] - y[, 1])^2 + (x[, 2] - y[, 2])^2)
    }
    n = nrow(x)
    m = nrow(y)
    dm = matrix(ncol = m, nrow = n)
    for (i in 1:n) {
      dm[i, ] = dfun(x[i, , drop = FALSE], y)
    }
    return(dm)
  }
  if (lonlat) {
    distfun <- distGeo
  }
  else {
    distfun <- distPlane
  }
  stopifnot(tr > 0)
  n <- round(n)
  stopifnot(n >= 1)
  if (inherits(fixed, "SpatialPoints")) 
    fixed <- coordinates(fixed)
  if (inherits(sample, "SpatialPoints")) 
    sample <- coordinates(sample)
  if (inherits(reference, "SpatialPoints")) 
    reference <- coordinates(reference)
  fixed <- as.matrix(fixed)[, 1:2]
  sample <- as.matrix(sample)[, 1:2]
  reference <- as.matrix(reference)[, 1:2]
  if (warn) {
    if (nrow(sample) < nrow(fixed)) {
      warning("nrow(sample) < nrow(fixed)")
    }
  }
  
  mindist <- function(distfun, a, b) {
    partition_count <- (1 %/% (100 / NROW(b))) + 1
    parts <- dismo::kfold(x=b, k=partition_count)
    r <- c()
    for(i in 1:partition_count) {
      mind <- apply(distfun(a, b[parts==i,]), 1, min)
      r <- cbind(r, mind)
    }
    apply(r, 1, min)
  }
  
  fromd <- mindist(distfun, fixed, reference) ##apply(distfun(fixed, reference), 1, min)
  tod <- mindist(distfun, sample, reference) ##apply(distfun(sample, reference), 1, min)
  ngb <- matrix(NA, nrow = length(fromd), ncol = n)
  iter <- sample(1:nrow(fixed))
  for (j in 1:n) {
    for (i in iter) {
      d <- abs(tod - fromd[i])
      if (min(d) < (tr * fromd[i])) {
        if (nearest) {
          x <- which.min(d)
        }
        else {
          x <- sample(which(d < (tr * fromd[i])), size = 1)
        }
        ngb[i, j] <- x
        tod[x] <- Inf
      }
    }
  }
  return(ngb)
}

and the ssb function then looks like this:

ssb <- function(p, a, reference, lonlat = TRUE, avg = TRUE) {
  distHaversine <- function(p1, p2) {
    r <- 6378137
    toRad <- pi/180
    p1 <- p1 * toRad
    p2 <- p2 * toRad
    p <- cbind(p1[, 1], p1[, 2], p2[, 1], p2[, 2])
    dLat <- (p[, 4] - p[, 2])
    dLon <- (p[, 3] - p[, 1])
    a <- sin(dLat/2) * sin(dLat/2) + cos(p[, 2]) * cos(p[, 
                                                         4]) * sin(dLon/2) * sin(dLon/2)
    dist <- 2 * atan2(sqrt(a), sqrt(1 - a)) * r
    as.vector(dist)
  }
  distGeo <- function(x, y) {
    n <- nrow(x)
    m <- nrow(y)
    dm <- matrix(ncol = m, nrow = n)
    for (i in 1:n) {
      dm[i, ] <- distHaversine(x[i, , drop = FALSE], y)
    }
    return(dm)
  }
  distPlane <- function(x, y) {
    dfun <- function(x, y) {
      sqrt((x[, 1] - y[, 1])^2 + (x[, 2] - y[, 2])^2)
    }
    n = nrow(x)
    m = nrow(y)
    dm = matrix(ncol = m, nrow = n)
    for (i in 1:n) {
      dm[i, ] = dfun(x[i, , drop = FALSE], y)
    }
    return(dm)
  }
  if (lonlat) {
    distfun <- distGeo
  }
  else {
    distfun <- distPlane
  }
  if (inherits(p, "SpatialPoints")) 
    p <- coordinates(p)
  if (inherits(a, "SpatialPoints")) 
    a <- coordinates(a)
  if (inherits(reference, "SpatialPoints")) 
    reference <- coordinates(reference)
  p <- as.matrix(p)[, 1:2]
  a <- as.matrix(a)[, 1:2]
  reference <- as.matrix(reference)[, 1:2]
  
  mindist <- function(distfun, a, b) {
    partition_count <- (1 %/% (1000 / NROW(b))) + 1
    parts <- dismo::kfold(x=b, k=partition_count)
    r <- c()
    for(i in 1:partition_count) {
      mind <- apply(distfun(a, b[parts==i,]), 1, min)
      r <- cbind(r, mind)
    }
    apply(r, 1, min)
  }
  pd <- mindist(distfun, p, reference) #   pd <- apply(distfun(p, reference), 1, min)
  ad <- mindist(distfun, a, reference) #   ad <- apply(distfun(a, reference), 1, min)
  if (avg) {
    res <- cbind(mean(pd), mean(ad))
    colnames(res) <- c("p", "a")
    return(res)
  }
  else {
    return(list(pd, ad))
  }
}

Spatially partioning lat long data in R with sperrorest partition.kmeans

2016-02-23T09:57:00.002+01:00

While trying out sperrorest::partition.kmeans I started wondering whether it did handle the date line (180, -180) correctly. A small clustering test confirmed that sperrorest doesn't take into account the spatial projection of the data and doesn't cluster together points from the same spatial area. To solve this I decided to convert my lat long data into cartesian coordinates and then call the partition.kmeans function.

library(sperrorest)

lonlat_xyz <- function(data, lonlat_cols) {
   rad_coordinates <- data[,lonlat_cols] * (pi/180)
   lon <- rad_coordinates[,lonlat_cols[1]]
   lat <- rad_coordinates[,lonlat_cols[2]]
   r <- cbind(cos(lat)*cos(lon), cos(lat)*sin(lon), sin(lat))
   colnames(r) <- c("x", "y", "z")
   return(r)
}

cartesian <- lonlat_xyz(data, lonlat) 
partitions <- sperrorest::partition.kmeans(cartesian, coords = c("x","y","z"))

Simple and lightweight C# REPL with Mono.CSharp

2015-11-24T22:22:00.000+01:00

Recently I needed a small C# REPL because I wanted to test some code on a computer where I couldn't install Visual Studio and the code I wanted to test used a lot of (Service) references so I couldn't use something like LINQPad. I first started with Roslyn but had some issues with using it from Visual Studio 2012 and as I didn't want to spend too much time on this problem I went for Mono.CSharp.Evaluator. Mono.CSharp can be installed with npm: Install-Package Mono.CSharp and only adds one reference which is a nice side-effect of using Mono.CSharp instead of the Roslyn scripting API.

Here is the full code (github) of my simple REPL. If you end a statement with ';' then it will run without returning an output, use this when you want to create objects . If you omit the ';' then the expression is evaluated and the result printed. Note that I added a reference to my program to the evaluator in order to be able to call the Factorial function. Also fon't to forget to run the necesssary Using statements.

// Install-Package Mono.CSharp
using System;
using System.Reflection;
using Mono.CSharp;

namespace SimpleREPL
{
  public class ExtraMath
  {
    public static int Factorial(int n)
    {
      // naive implementation but fast enough for small n
      int result = 1;
      for (int i = 1; i <= n; i++)
      {
        result *= i;
      }
      return result;
    }
  }

  internal class Program
  {
    private static void Main(string[] args)
    {
      Console.WriteLine("Starting Simple C# REPL, enter q to quit");
      var evaluator = new Evaluator(new CompilerContext(
        new CompilerSettings(),
        new ConsoleReportPrinter()));
      evaluator.ReferenceAssembly(Assembly.GetExecutingAssembly());
      evaluator.Run("using System;");
      evaluator.Run("using SimpleREPL;");
      while (true)
      {
        Console.Write("> ");
        var input = Console.ReadLine();
        input = input.TrimStart('>', ' ');
        if (input.ToLower() == "q")
        {
          return;
        }
        try
        {
          if (input.EndsWith(";"))
          {
            evaluator.Run(input);
          }
          else
          {
            var output = evaluator.Evaluate(input);
            Console.WriteLine(output);
          }
        }
        catch
        {
          Console.WriteLine("Error in input");
        }
      }
    }
  }
}

Weighted Relative Neighborhood Graph in R based on cccd::rng

2015-11-11T11:11:00.000+01:00

The R package cccd contains a nice implementation of the Relative Neighborhood Graph (rng) but in the current version 1.5 it returns a non-weighted igraph. But for one of my experiments I needed the weighted version so I've slightly changed the code to get an igraph with weights.

rng <- function (x = NULL, dx = NULL, r = 1, method = NULL, usedeldir = TRUE,
          open = TRUE, k = NA, algorithm = "cover_tree", weighted = TRUE) {
  if (is.na(k)) {
    if (is.null(dx)) {
      if (is.null(x))
        stop("One of x or dx must be given.")
      dx <- as.matrix(proxy::dist(x, method = method))
    }
    else {
      usedeldir <- FALSE
    }
    n <- nrow(dx)
    A <- matrix(0, nrow = n, ncol = n)
    if (is.vector(x))
      x <- matrix(x, ncol = 1)
    if (usedeldir && ncol(x) == 2) {
      del <- deldir::deldir(x[, 1], x[, 2])
      for (edge in 1:nrow(del$delsgs)) {
        i <- del$delsgs[edge, 5]
        j <- del$delsgs[edge, 6]
        d <- min(apply(cbind(dx[i, -c(i, j)], dx[j, -c(i, j)]), 1, max))
        rd <- r * dx[i, j]
        if ((open && rd < d) || rd <= d) {
          A[i, j] <- A[j, i] <- rd
        }
      }
    } else {
      diag(dx) <- Inf
      for (i in 1:n) {
        for (j in setdiff(1:n, i)) {
          d <- min(apply(cbind(dx[i, -c(i, j)], dx[j, -c(i, j)]), 1, max))
          rd <- r * dx[i, j]
          if ((open && rd < d) || rd <= d) {
            A[i, j] <- A[j, i] <- rd
          }
        }
      }
    }
    diag(A) <- 0
    out <- graph.adjacency(A, mode = "undirected", weighted = weighted)
  } else {
    if (is.null(x))
      stop("x must not be null")
    n <- nrow(x)
    k <- min(k, n - 1)
    dx <- get.knn(x, k = k, algorithm = algorithm)
    edges <- NULL
    weights <- NULL
    for (i in 1:n) {
      i.indices <- dx$nn.index[i, ]
      i.dists <- dx$nn.dist[i, ]
      for (j in 1:k) {
        rd <- r * i.dists[j]/2
        j.indices <- dx$nn.index[i.indices[j], ]
        j.dists <- dx$nn.dist[i.indices[j], ]
        rd <- r * i.dists[j]
        S <- setdiff(intersect(i.indices, j.indices),
                     c(i, i.indices[j]))
        if (length(S) > 0) {
          d <- Inf
          for (si in S) {
            a <- which(i.indices == si)
            b <- which(j.indices == si)
            d <- min(d, max(i.dists[a], j.dists[b]))
          }
          if ((open && rd < d) || rd <= d) {
            edges <- cbind(edges, c(i, i.indices[j]))
            weights <- cbind(weights, rd)
          }
        }
      }
    }
    g <- graph(edges, n = n, directed = FALSE)
    if( weighted ) {
      edge.attributes(g) <- list(weight=weights)
    }
    out <- simplify(g, edge.attr.comb = "first")
  }
  if (!is.null(x)) {
    out$layout <- x
  }
  out$r <- r
  out
}

Five Things We Need to Know About Technological Change by Neil Postman

2015-10-22T21:12:00.000+02:00

What follows are my notes and highlighted sections from an excellent essay from Neil Postman. My comments are in italic.
Source: http://web.cs.ucdavis.edu/~rogaway/classes/188/materials/postman.pdf.

Five Things We Need to Know About Technological Change

First Idea

The first idea is that all technological change is a trade-off.

This means that for every advantage a new technology offers, there is always a corresponding disadvantage. The disadvantage may exceed in importance the advantage, or the advantage may well be worth the cost.

note: cost and benefits are different for different persons
action: reflect on cost and benefits of the technology you use (plane, car, computer, smartphone, specific websites, webshops, apps, food tech, health tech, ebooks, "free" books/pdfs, skype, telephone, messenger/sms, ...)

Perhaps the best way I can express this idea is to say that the question, “What will a new technology do?” is no more important than the question, “What will a new technology undo?” Indeed, the latter question is more important, precisely because it is asked so infrequently. One might say, then, that a sophisticated perspective on technological change includes one’s being skeptical of Utopian and Messianic visions drawn by those who have no sense of history or of the precarious balances on which culture depends. In fact, if it were up to me, I would forbid anyone from talking about the new information technologies unless the person can demonstrate that he or she knows something about the social and psychic effects of the alphabet, the mechanical clock, the printing press, and telegraphy. In other words, knows something about the costs of great technologies.

Idea Number One, then, is that culture always pays a price for technology.

Second Idea

This leads to the second idea, which is that the advantages and disadvantages of new technologies are never distributed evenly among the population. This means that every new technology benefits some and harms others. There are even some who are not affected at all.

The questions, then, that are never far from the mind of a person who is knowledgeable about technological change are these: Who specifically benefits from the development of a new technology? Which groups, what type of person, what kind of industry will be favored? And, of course, which groups of people will thereby be harmed?

That is why it is always necessary for us to ask of those who speak enthusiastically of computer technology, why do you do this? What interests do you represent? To whom are you hoping to give power? From whom will you be withholding power?

I do not mean to attribute unsavory, let alone sinister motives to anyone. I say only that since technology favors some people and harms others, these are questions that must always be asked. And so, that there are always winners and losers in technological change is the second idea.

Third Idea

Embedded in every technology there is a powerful idea, sometimes two or three powerful ideas. These ideas are often hidden from our view because they are of a somewhat abstract nature. But this should not be taken to mean that they do not have practical consequences.

Perhaps you are familiar with the old adage that says: To a man with a hammer, everything looks like a nail. We may extend that truism: To a person with a pencil, everything looks like a sentence. To a person with a TV camera, everything looks like an image. To a person with a computer, everything looks like data. I do not think we need to take these aphorisms literally. But what they call to our attention is that every technology has a prejudice. Like language itself, it predisposes us to favor and value certain perspectives and accomplishments. In a culture without writing, human memory is of the greatest importance, as are the proverbs, sayings and songs which contain the accumulated oral wisdom of centuries.... The television person values immediacy, not history. And computer people, what shall we say of them? Perhaps we can say that the computer person values information, not knowledge, certainly not wisdom. Indeed, in the computer age, the concept of wisdom may vanish altogether.

note: see also Nicolas Carr (impact of Google on thinking/memory)

The third idea, then, is that every technology has a philosophy which is given expression in how the technology makes people use their minds, in what it makes us do with our bodies, in how it codifies the world, in which of our senses it amplifies, in which of our emotional and intellectual tendencies it disregards. This idea is the sum and substance of what the great Catholic prophet, Marshall McLuhan meant when he coined the famous sentence, “The medium is the message.”

Fourth Idea

Technological change is not additive; it is ecological.

A new medium does not add something; it changes everything. In the year 1500, after the printing press was invented, you did not have old Europe plus the printing press. You had a different Europe. After television, America was not America plus television. Television gave a new coloration to every political campaign, to every home, to every school, to every church, to every industry, and so on.

That is why we must be cautious about technological innovation. The consequences of technological change are always vast, often unpredictable and largely irreversible. That is also why we must be suspicious of capitalists. Capitalists are by definition not only personal risk takers but, more to the point, cultural risk takers. The most creative and daring of them hope to exploit new technologies to the fullest, and do not much care what traditions are overthrown in the process or whether or not a culture is prepared to function without such traditions. Capitalists are, in a word, radicals.

Fifth Idea

I come now to the fifth and final idea, which is that media tend to become mythic. I use this word in the sense in which it was used by the French literary critic, Roland Barthes. He used the word “myth” to refer to a common tendency to think of our technological creations as if they were God-given, as if they were a part of the natural order of things. I have on occasion asked my students if they know when the alphabet was invented. The question astonishes them. It is as if I asked them when clouds and trees were invented. The alphabet, they believe, was not something that was invented. It just is. It is this way with many products of human culture but with none more consistently than technology. Cars, planes, TV, movies, newspapers—they have achieved mythic status because they are perceived as gifts of nature, not as artifacts produced in a specific political and historical context.

When a technology become mythic, it is always dangerous because it is then accepted as it is, and is
therefore not easily susceptible to modification or control.

What I am saying is that our enthusiasm for technology can turn into a form of idolatry and our belief in its beneficence can be a false absolute. The best way to view technology is as a strange intruder, to remember that technology is not part of God’s plan but a product of human creativity and hubris, and that its capacity for good or evil rests entirely on human awareness of what it does for us and to us.

Conclusion

And so, these are my five ideas about technological change. First, that we always pay a price for technology; the greater the technology, the greater the price. Second, that there are always winners and losers, and that the winners always try to persuade the losers that they are really winners. Third, that there is embedded in every great technology an epistemological, political or social prejudice. Sometimes that bias is greatly to our advantage. Sometimes it is not. The printing press annihilated the oral tradition; telegraphy annihilated space; television has humiliated the word; the computer, perhaps, will degrade community life. And so on. Fourth, technological change is not additive; it is ecological, which means, it changes everything and is, therefore, too important to be left entirely in the hands of Bill Gates. And fifth, technology tends to become mythic; that is, perceived as part of the natural order of things, and therefore tends to control more of our lives than is good for us.

If we had more time, I could supply some additional important things about technological change but I will stand by these for the moment, and will close with this thought. In the past, we experienced technological change in the manner of sleep-walkers. Our unspoken slogan has been “technology über alles,” and we have been willing to shape our lives to fit the requirements of technology, not the requirements of culture. This is a form of stupidity, especially in an age of vast technological change. We need to proceed with our eyes wide open so that we may use technology rather than be used by it.

Workaround object 'couleursIn' not found in R package eVenn version 2.2

2015-10-06T11:26:00.000+02:00

When trying to define a venn diagram with custom colors in the eVenn diagram:

set.seed(42)
d <- data.frame(a = sample(rep(c(0,1), 20)), 
                b = sample(rep(c(0,1), 20)), 
                c = sample(rep(c(0,1), 20)))
evenn(matLists=as.matrix(d), display = TRUE, couleurs = c("#90BA6E","#956EAD","#9F5845"))

I encountered the following error:

Error: object 'couleursIn' not found

This error only occurs when you pass a vector with multiple colors to couleurs and you've set Solid = TRUE, which is the default value. First thing to do after getting the error is calling dev.off() as the error will have left a connection to the png file open. To fix the error you have to define a variable couleursIn with colors with alpha values if you want to create similar venn diagrams as the default ones but with different colors:

couleursIn <- c("#90BA6E80","#956EAD80","#9F584580")
evenn(matLists=as.matrix(d), display = TRUE, couleurs = c("#90BA6E","#956EAD","#9F5845"))

Note the 80 at the end of each color which stands for an alpha value of 128 which is 50%. This can be checked with col2rgb("#90BA6E80", alpha = TRUE).

Workaround missing support for as.formula in R package maxlike version 0.1-5

2015-09-24T14:58:00.000+02:00

Today I noticed that the predict function for maxlike didn't support models build with formulas from "as.formula". The error I got was:

Error in predict.maxlikeFit(model, data.frame(p), ...) :
at least 1 covariate in the formula is not in rasters.

I send in a pull request but in the mean time you can workaround this problem with the following wrapper for the maxlike function:

## workaround missing support for as.formula in maxlike
maxlike <- function(formula, ...) {
  m <- maxlike::maxlike(formula, ...)
  m$call$formula <- formula
  return(m)
}

Workaround "n.trees" is missing in R package gbm version 2.1.1

2015-09-24T13:19:00.002+02:00

Just short note to anyone encountering following error in the gbm package version 2.1.1:

Error in paste("Using", n.trees, "trees...\n") :
argument "n.trees" is missing, with no default

This issue will be fixed in the next version but in the mean time I'll post my workaround. If you add following function to your code then everything should work:

## work around bug in gbm 2.1.1
predict.gbm <- function (object, newdata, n.trees, type = "link", single.tree = FALSE, ...) {
  if (missing(n.trees)) {
    if (object$train.fraction < 1) {
      n.trees <- gbm.perf(object, method = "test", plot.it = FALSE)
    }
    else if (!is.null(object$cv.error)) {
      n.trees <- gbm.perf(object, method = "cv", plot.it = FALSE)
    }
    else {
      n.trees <- length(object$train.error)
    }
    cat(paste("Using", n.trees, "trees...\n"))
    gbm::predict.gbm(object, newdata, n.trees, type, single.tree, ...)
  }
}

Notes and reflections on From here to human-level AI

2015-03-19T20:00:00.000+01:00

The Ghent strong AI meetup group will discuss on the 20th of march the 2007 paper from John McCarthy From here to human-level AI so I decided to highlight some parts and write-up my notes and thoughts.

1. What is human-level AI

There are two approaches to human-level AI, but each presents difficulties. It isn’t a question of deciding between them, because each should eventually succeed; it is more a race.

If we understood enough about how the human intellect works, we could simulate it.

To the extent that we understand the problems achieving goals in the world presents to intelligence we can write intelligent programs. That's what this article is about.
Much of the public recognition of AI has been for programs with a little bit of AI and a lot of computing.

There are some big projects and a lot of researchers attacking the human-level AI problem with one of the two approaches. The first approach, or at least part of it, is used by the American BRAIN Initiative (Brain Research through Advancing Innovative Neurotechnologies) and the European Human Brain Project. The second approach and especially everything related to deep learning has recently had a fair amount of publicity due to human-level and super-human results on some pattern recognition tasks (image classification, face verification) and games (human-level performance on 29 Atari games). For an overview and history of deep learning you can check out the 88 page overview article from Jürgen Schmidhuber. As a side note Jürgen Schmidhuber recently did an interesting AMA (ask me anything) on Reddit (summary from FastML).

2. The common sense informatic situation

The key to reaching human-level AI is making systems that operate successfully in the common sense informatic situation.
In general a thinking human is in what we call the common sense informatic situation. It is more general than any bounded informatic situation. The known facts are incomplete, and there is no a priori limitation on what facts are relevant. It may not even be decided in advance what phenomena are to be taken into account. The consequences of actions cannot be fully determined. The common sense informatic situation necessitates the use of approximate concepts that cannot be fully defined and the use of approximate theories involving them. It also requires nonmonotonic reasoning in reaching conclusions.

Nonmonotonic reasoning = a logic is non-monotonic if some conclusions can be invalidated by adding more knowledge (source).

Common sense facts and common sense reasoning are necessarily imprecise. The imprecision necessitated by the common sense informatic situation applies to computer programs as well as to people.

3. The use of mathematical logic

Mathematical logic was devised to formalize precise facts and correct reasoning. Its founders, Leibniz, Boole and Frege, hoped to use it for common sense facts and reasoning, not realizing that the imprecision of concepts used in common sense language was often a necessary feature and not always a bug. The biggest success of mathematical logic was in formalizing purely mathematical theories for which imprecise concepts are unneeded. Since the common sense informatic situation requires using imprecise facts and imprecise reasoning, the use of mathematical logic for common sense has had limited success. This has caused many people to give up. Others devise extended logical languages and even extended forms of mathematical logic.

Further on he notes that using different concepts and different predicate and function symbols in a new mathematical logic language might still make mathematical logic adequate for expressing common sense. But he is not very optimistic.

Success so far has been moderate, and it isn’t clear whether greater success can be obtained by changing the concepts and their representation by predicate and function symbols or by varying the nonmonotonic formalism.

4. Approximate concepts and approximate theories

Other kinds of imprecision are more fundamental for intelligence than numerical imprecision. Many phenomena in the world are appropriately described in terms of approximate concepts. Although the concepts are imprecise, many statements using them have precise truth values

He follows up with two clarifying examples, one about the concept Mount Everest, the other about the concept welfare of a chicken. Mount Everest is an approximate concept because the exact pieces of rock and ice that constitute it are unclear. But it is still possible to infer solid conclusions based on a foundation built on a quicksand of approximate concepts without definite extensions e.g. if you haven't been to Asia then you've never climbed Mount Everest. The core of the welfare of a chicken problem is: is it better to raise a chicken with care and nice food and then slaughter it or would it have a better life in the wild, risking starvation and foxes ? McCarthy concludes from this:

There is no truth of the matter to be determined by careful investigation of chickens. When a concept is inherently approximate, it is a waste of time to try to give it a precise definition.

In order to reach human-level AI we'll have to be able to approximate concepts in a way that the computer can reason about them.

5. Nonmonotonic reasoning

6. Elaboration tolerance

7. Formalization of context

8. Reasoning about events - especially action

Human level intelligence requires reasoning about strategies of action, i.e. action programs. It also requires considering multiple actors and also concurrent events and continuous events. Clearly we have a long way to go.

9. Introspection and self-awareness

People have a limited ability to observe their own mental processes. For many intellectual tasks introspection is irrelevant. However, it is at least relevant for evaluating how one is using one’s own thinking time. Human-level AI will require introspective ability. In fact programs can have more than humans do, because they can examine themselves, both in source and compiled form and also reason about the current values of the variables in the program.

10. Heuristics

The largest qualitative gap between human performance and computer performance is in the area of heuristics, even though the gap is disguised in many applications by the millions-fold speed advantage of computers. The general purpose theorem proving programs run very slowly, and the special purpose programs are very specialized in their heuristics.
I think the problem lies in our present inability to give programs domain and problem dependent heuristic advice.

McCarthy advocates the usage of declarative heuristics and explains the concept of postponable variables in constraint satisfaction problems.

11. Psychological, social and political obstacles

In this article, McCarthy states that although the main problems in reaching human-level AI lay in the inherent difficulty of the scientific problems, research is hampered by the focus of the computer science world to connecting the basic research to applied problems. The artificial intelligence has encountered philosophical and ideological (religious) objections but the attacks on AI have been fairly limited.

As the general public gets more and more acquainted with the potential dangers, of human-level AI and especially super-human level AI, I believe, that the pressure against AI research will increase.

An interesting book by Nick Bostrom covering the dangers of AI is Superintelligence: Paths, Dangers, Strategies.

12. Conclusion

Between us and human-level intelligence lie many problems. They can be summarized as that of succeeding in the common sense informatic situation.

If you want to read more about the road to human-level AI and superintelligence and its possible consequences then I can recommend this 2 part article by Tim Urban on his blog Wait But Why.

Benchmarking reading binary values from a file with F#, Python, Julia, R, Go and OCaml

2014-07-22T21:51:00.000+02:00

In one of my recent posts I showed some F# code for storing and reading integers as binary data. Since then I've created different versions of the reading part in Python, Julia, Go, OCaml and R to be able to compare the performance of the different languages on this simple task and to get a feeling on what it's like to program in Julia, Go and OCaml as I hadn't created any programs in these languages yet.

Below table shows the benchmark results for reading 10 times 10000 values and 10000 times 10 values with links to the source code files in the different programming languages:

Language	10 times 10000 values	10000 times 10 values
F#	6 seconds	20 seconds
Python	26 seconds	40 seconds
Julia	45 seconds	72 seconds
Go	8 seconds	25 seconds
OCaml	2.5 seconds	48 seconds
R	110 seconds	NA

The overall fastest version is the one written in F# but note that it's also the version I have tweaked the most. As I'm not very experienced in most of the languages so any performance tips are welcome. Note that I tried using memory mapped files in .NET and Python this improved performance when querying lots of values from the same file but also made it worse in other cases.

The implementation of the functionality is most of the times rather similar in the different languages. Some notable differences where:

Julia apparently doesn't have a null value so I refrained from checking whether the read integer value was equal to the int32 minimum value (-2147483648).
In Go converting the bytes to integers was faster with a custom function.
I didn't find a function in the OCaml Core library to convert bytes to a 32-bit integer, but luckily I found one on Stack Overflow.

F#:

open System
open System.IO

let readValue (reader:BinaryReader) cellIndex = 
    reader.BaseStream.Seek(int64 (cellIndex*4), SeekOrigin.Begin) |> ignore
    match reader.ReadInt32() with
    | Int32.MinValue -> None
    | v -> Some(v)
        
let readValues indices fileName = 
    use reader = new BinaryReader(File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.Read))
    let values = Array.map (readValue reader) indices
    values

Python:

def read_values(filename, indices):
    # indices are sorted and unique
    values = []
    with open(filename, 'rb') as f:
        for index in indices:
            f.seek(index*4L, os.SEEK_SET)
            b = f.read(4)
            v = struct.unpack("@i", b)[0]
            if v == -2147483648:
                v = None
            values.append(v)
    return values

Julia:

function readvalue(stream, position)
    seek(stream, position)
    return read(stream, Int32)
end

function readvalues(filename::String, indices)
    stream = open(filename, "r")
    try
        return Int32[readvalue(stream, index*4) for index in indices]
    finally
        close(stream)
    end
end

Go:

import ("os")
func bytes2int(b []byte) int32 {
    v := int32(0)
    for  i := 0; i < 4; i++ {
        v = v | (int32(b[i]) << (uint(8*i)))
    }
    return v
}

func readValues(indices []int, filename string) []int32 {
    results := make([]int32, len(indices))
    b := make([]byte, 4)
    f,_ := os.Open(filename)
    for i, cellIndex := range indices {
        f.Seek(int64(cellIndex*4), os.SEEK_SET)
        f.Read(b)
        value := bytes2int(b) // around 10-20% faster then binary.Read
        if value != -2147483648 {
            results[i] = value
        } else {
            results[i] = 99999
        }
    }
    return results
}

OCaml:

let input_le_int32 inchannel = (* http://stackoverflow.com/a/6031286/477367 *)
  let res = ref 0l in
    for i = 0 to 3 do
      let byte = input_byte inchannel in
        res := Int32.logor !res (Int32.shift_left (Int32.of_int byte) (8*i))
    done;

    match !res with
      | -2147483648l -> None
      | v -> Some(v)

let readvalue inchannel index =
  seek_in inchannel (index*4);
  input_le_int32 inchannel

let readvalues (indices:int array) filename =
  let inchannel = open_in_bin filename in
    try
      let result = Array.map (readvalue inchannel) indices in
        close_in inchannel;
        result
    with e ->
      close_in_noerr inchannel;
      raise e

read.values <- function(filename, indices) {
  conn <- file(filename, "rb")
  read.value <- function(index) {
    seek(conn, where=index*4)
    readBin(conn, integer(), size = 4, n = 1, endian = "little")
  }
  r <- sapply(indices,read.value)
  close(conn)
  r[r==-2147483648] <- NA
  r
}

Any suggestions for improving the implementation in one of the above programming languages ? Which language would you like to compare my results with ? Which other language(s) do you expect to be faster than my benchmark results ? Can you help me out with a version in your favorite language or in C, fortran, Common Lisp, Scheme, Clojure, Java or J ?

Becoming Functional

2014-07-11T15:00:00.000+02:00

Here is my first book review from the O'Reilly Reader Review Program. I picked Becoming Functional by Joshua Backfield because I thought it would be nice to compare it to the latest book I've read on Functional Programming in JavaScript by Michael Fogus (accidentally also published by O'Reilly).

A step by step introduction to some basic concepts of functional programming like higher order functions (passing a function to a method), pure functions (for the same input always return the same output without side effects), immutable variables, recursion and lazy evaluation.

The first chapters are in Java but starting from chapter 4 the book switches to Groovy and the latest chapters are in Scala. The Java code is not very complicated and is probably understandable for anyone with knowledge of statically typed object-oriented programming languages. Note that all Java samples use Java 7 which is rather verbose for functional programming which is one of the reasons the book switches to Groovy and Scala. The other reason is off course is that these languages support functional concepts out of the box.

Content wise the books starts with an introduction to functional programming then makes the transition from first-class functions to higher order functions and pure functions. Next are immutable variables and recursion, with a nice introduction to tail recursion. Followed by small side steps to laziness and statements. To conclude with pattern matching functional object-oriented programming. The last chapter is surprisingly as it first gives advice on transitioning into functional programming then talks about new design patterns and ends with a full implementation of a simplistic database in Scala.

It's not a bad book in the sense that it teaches incorrect material but it's not a very good book neither. It's a book written for Java developers that have never heard of the basic concepts of functional programming but personally I prefer a different approach to achieve this goal. Instead of refactoring code from the imperative style into the functional style and along the way introducing concepts, I think it's faster and easier to learn functional programming by clearly explaining and showing the concepts directly. Only then you should elaborate on how to migrate from the original code to the functional style and explain why this is a good thing to do. Although it's a rather short book (an estimated 140 pages) it is not very dense but it gives semi-real world example based on requirements from a fictional company XXY. Only the most common functional programming concepts are introduced.

My advice is to buy this book if you're a Java developer who has no experience in functional programming.

Otherwise I would skip this book and instead read:

Real World Functional Programming: examples in C# (which is a better version of Java, at least for functional programming) and F# (programming language from the ML-family)
Functional Programming in JavaScript
or any book on Erlang, Haskell, Clojure (e.g. Joy of Clojure), Common Lisp, F#, OCaml, Scala, ...

Or take a look at my page of recommendations.