Elesticsearch is not able to index the geo_shape defined in the 9 missing records
During testing for a –ve scenario we ingested the same data for the kind : opendes:staticrela:states:4.0.0 which lead to the updation of the records.
Hence to recreate the scenario, I created a new schema with kind : opendes:relationships:teststates:1.0.0
Reingested the file with the metadata id : opendes:dataset--File.Generic:6645b7c0-821c-4c32-91dc-cb985a6707fd
Runid for the shapefile_ingestor_wf : 17c0935d-9562-4274-ae7b-54f5285ce792, which ingested 49 records, logs attached.
However the search service gives only 40 records in the output :
{ "kind": "opendes:relationships:teststates:1.0.0", "returnedFields": [ "id" ], "limit": 100 }
Response :
{ "cursor": "932F1FA4A28082E6FC5F97FE4D4BC102", "results": [ { "id": "opendes:teststates:relationships-YXJpem9uYQ" }, { "id": "opendes:teststates:relationships-Y29sb3JhZG8" }, { "id": "opendes:teststates:relationships-bWFyeWxhbmQ" }, { "id": "opendes:teststates:relationships-dGV4YXM" }, { "id": "opendes:teststates:relationships-bm9ydGggY2Fyb2xpbmE" }, { "id": "opendes:teststates:relationships-bmV3IG1leGljbw" }, { "id": "opendes:teststates:relationships-Z2VvcmdpYQ" }, { "id": "opendes:teststates:relationships-ZGlzdHJpY3Qgb2YgY29sdW1iaWE" }, { "id": "opendes:teststates:relationships-d2VzdCB2aXJnaW5pYQ" }, { "id": "opendes:teststates:relationships-c291dGggY2Fyb2xpbmE" }, { "id": "opendes:teststates:relationships-dmlyZ2luaWE" }, { "id": "opendes:teststates:relationships-a2Fuc2Fz" }, { "id": "opendes:teststates:relationships-b2tsYWhvbWE" }, { "id": "opendes:teststates:relationships-ZGVsYXdhcmU" }, { "id": "opendes:teststates:relationships-a2VudHVja3k" }, { "id": "opendes:teststates:relationships-YWxhYmFtYQ" }, { "id": "opendes:teststates:relationships-bWFzc2FjaHVzZXR0cw" }, { "id": "opendes:teststates:relationships-d3lvbWluZw" }, { "id": "opendes:teststates:relationships-b3JlZ29u" }, { "id": "opendes:teststates:relationships-aWRhaG8" }, { "id": "opendes:teststates:relationships-bmVicmFza2E" }, { "id": "opendes:teststates:relationships-bm9ydGggZGFrb3Rh" }, { "id": "opendes:teststates:relationships-bWFpbmU" }, { "id": "opendes:teststates:relationships-ZmxvcmlkYQ" }, { "id": "opendes:teststates:relationships-bmV3IGhhbXBzaGlyZQ" }, { "id": "opendes:teststates:relationships-c291dGggZGFrb3Rh" }, { "id": "opendes:teststates:relationships-cGVubnN5bHZhbmlh" }, { "id": "opendes:teststates:relationships-aW93YQ" }, { "id": "opendes:teststates:relationships-bW9udGFuYQ" }, { "id": "opendes:teststates:relationships-dmVybW9udA" }, { "id": "opendes:teststates:relationships-bmV3IHlvcms" }, { "id": "opendes:teststates:relationships-bmV3IGplcnNleQ" }, { "id": "opendes:teststates:relationships-Y29ubmVjdGljdXQ" }, { "id": "opendes:teststates:relationships-cmhvZGUgaXNsYW5k" }, { "id": "opendes:teststates:relationships-b2hpbw" }, { "id": "opendes:teststates:relationships-d2FzaGluZ3Rvbg" }, { "id": "opendes:teststates:relationships-dXRhaA" }, { "id": "opendes:teststates:relationships-bmV2YWRh" }, { "id": "opendes:teststates:relationships-aW5kaWFuYQ" }, { "id": "opendes:teststates:relationships-Y2FsaWZvcm5pYQ" } ], "totalCount": 40 }
When I validated the information found these 9 ids missing again :
opendes:teststates:relationships-aWxsaW5vaXM opendes:teststates:relationships-bWlzc291cmk opendes:teststates:relationships-dGVubmVzc2Vl opendes:teststates:relationships-bWlzc2lzc2lwcGk opendes:teststates:relationships-YXJrYW5zYXM opendes:teststates:relationships-bG91aXNpYW5h opendes:teststates:relationships-bWljaGlnYW4 opendes:teststates:relationships-d2lzY29uc2lu opendes:teststates:relationships-bWlubmVzb3Rh
Tried to get it one of them from storage :
{ "data": { "STATE_NAME": "Illinois", "STATE_FIPS": 17, "SUB_REGION": "E N Cen", "STATE_ABBR": "slb:wing:ABBR-sxs0f1a5219-4640-50af-9f63-c09140d57c4d:", "LAND_KM": 143986.61, "WATER_KM": 1993.335, "PERSONS": 11430602, "FAMILIES": 2924880, "HOUSHOLD": 4202240, "MALE": 5552233, "FEMALE": 5878369, "WORKERS": "4199206.0", "DRVALONE": "3741715.0", "CARPOOL": "652603.0", "PUBTRANS": "538071.0", "EMPLOYED": "5417967.0", "UNEMPLOY": "385040.0", "SERVICE": "1360159.0", "MANUAL": "828906.0", "P_MALE": "0.486", "P_FEMALE": "0.514", "SAMP_POP": "1747776.0", "SpatialLocation": { "AsIngestedCoordinates": { "type": "AnyCrsFeatureCollection", "features": [ { "type": "AnyCrsFeature", "properties": {}, "geometry": { "type": "AnyCrsPolygon", "coordinates": [ [ [ 37.51099000000001, -88.071564 ], [ 37.583572000000004, -88.134171 ], [ 37.628479, -88.157631 ], [ 37.660686, -88.15937 ], [ 37.700745, -88.133636 ], [ 37.735400999999996, -88.072472 ], [ 37.805683, -88.035576 ], [ 37.817612, -88.086029 ], [ 37.831249, -88.089264 ], [ 37.827522, -88.042137 ], [ 37.843745999999996, -88.034241 ], [ 37.867808999999994, -88.075737 ], [ 37.895306000000005, -88.101456 ], [ 37.90617, -88.100082 ], [ 37.896004000000005, -88.044868 ], [ 37.905758000000006, -88.026588 ], [ 37.917591, -88.030441 ], [ 37.92366, -88.084 ], [ 37.944, -88.078941 ], [ 37.929783, -88.064621 ], [ 37.934498000000005, -88.041771 ], [ 37.956264000000004, -88.042511 ], [ 37.975055999999995, -88.021706 ], [ 38.00823600000001, -88.029213 ], [ 38.03353100000001, -88.021698 ], [ 38.03830300000001, -88.041473 ], [ 38.04512, -88.043091 ], [ 38.054084999999986, -88.034729 ], [ 38.073307, -87.975296 ], [ 38.09674799999999, -87.964867 ], [ 38.09234599999999, -88.012329 ], [ 38.10330200000001, -88.018547 ], [ 38.131760000000014, -87.973503 ], [ 38.13691299999999, -87.950569 ], [ 38.15752800000001, -87.931992 ], [ 38.171131, -87.932289 ], [ 38.200714000000005, -87.977928 ], [ 38.234814, -87.986008 ], [ 38.241085, -87.980019 ], [ 38.30477099999999, -87.925919 ], [ 38.302345, -87.913651 ], [ 38.281048, -87.914108 ], [ 38.300658999999996, -87.888466 ], [ 38.315552, -87.883446 ], [ 38.316788, -87.874039 ], [ 38.28536199999999, -87.863007 ], [ 38.28609800000001, -87.850082 ], [ 38.35252399999999, -87.834503 ], [ 38.378124000000014, -87.784019 ], [ 38.41796500000001, -87.748428 ], [ 38.44548, -87.738953 ], [ 38.45709600000001, -87.758659 ], [ 38.466125000000005, -87.756096 ], [ 38.48153300000001, -87.692818 ], [ 38.50400500000001, -87.679909 ], [ 38.50044299999999, -87.653534 ], [ 38.51536899999999, -87.65139 ], [ 38.54742400000001, -87.672943 ], [ 38.573871999999994, -87.652855 ], [ 38.593177999999995, -87.640594 ], [ 38.599209, -87.619827 ], [ 38.622917, -87.628647 ], [ 38.642810999999995, -87.625191 ], [ 38.672169, -87.588478 ], [ 38.68597399999999, -87.543892 ], [ 38.73663300000001, -87.508316 ], [ 38.769722, -87.508003 ], [ 38.77669900000001, -87.519028 ], [ 38.795559, -87.507889 ], [ 38.857890999999995, -87.550507 ], [ 38.869811999999996, -87.559059 ], [ 38.90486100000001, -87.5392 ], [ 38.93191899999999, -87.530182 ], [ 38.96370300000001, -87.53347 ], [ 38.97707700000001, -87.547905 ], [ 38.99408299999999, -87.591858 ], [ 38.995743000000004, -87.581749 ], [ 39.062434999999994, -87.58532 ], [ 39.08460600000001, -87.612007 ], [ 39.08897400000001, -87.630867 ], [ 39.10394299999999, -87.631668 ], [ 39.11346800000001, -87.662262 ], [ 39.130652999999995, -87.659454 ], [ 39.146679000000006, -87.670326 ], [ 39.168507000000005, -87.644257 ], [ 39.196068, -87.607925 ], [ 39.198128, -87.594208 ], [ 39.20846599999999, -87.588593 ], [ 39.248752999999994, -87.584564 ], [ 39.258162999999996, -87.606895 ], [ 39.281418, -87.615799 ], [ 39.297661000000005, -87.610619 ], [ 39.30740399999999, -87.625237 ], [ 39.338268, -87.597664 ], [ 39.350525000000005, -87.540215 ], [ 39.47744800000001, -87.538567 ], [ 39.609341, -87.535576 ], [ 39.887302000000005, -87.535774 ], [ 40.16619499999999, -87.535339 ], [ 40.48324600000001, -87.535675 ], [ 40.494609999999994, -87.53717 ], [ 40.74541099999999, -87.532669 ], [ 41.00993, -87.532021 ], [ 41.173756, -87.531731 ], [ 41.30130399999999, -87.532448 ], [ 41.46971500000001, -87.532646 ], [ 41.723591, -87.529861 ], [ 41.847331999999994, -87.612625 ], [ 42.059822, -87.670547 ], [ 42.15645599999999, -87.760239 ], [ 42.314212999999995, -87.836945 ], [ 42.48913200000001, -87.79731 ], [ 42.48961299999999, -88.194702 ], [ 42.49197000000001, -88.297897 ], [ 42.489655, -88.70652 ], [ 42.490905999999995, -88.764954 ], [ 42.49086399999999, -88.939079 ], [ 42.497906, -89.359444 ], [ 42.49749, -89.400497 ], [ 42.50345999999999, -89.834618 ], [ 42.504108, -89.923569 ], [ 42.508362000000005, -90.419975 ], [ 42.50936100000001, -90.638329 ], [ 42.494698, -90.651772 ], [ 42.47564299999999, -90.648346 ], [ 42.46055999999999, -90.605827 ], [ 42.42183700000001, -90.563583 ], [ 42.38878299999999, -90.491043 ], [ 42.360073, -90.441597 ], [ 42.340633, -90.427681 ], [ 42.263924, -90.417984 ], [ 42.24264500000001, -90.407173 ], [ 42.21020899999999, -90.367729 ], [ 42.19731899999999, -90.323601 ], [ 42.15972099999999, -90.230934 ], [ 42.12268800000001, -90.191574 ], [ 42.12050199999999, -90.176086 ], [ 42.103745, -90.166649 ], [ 42.06104300000001, -90.168098 ], [ 42.03342799999999, -90.150536 ], [ 41.98396299999999, -90.14267 ], [ 41.93077500000001, -90.154518 ], [ 41.80613700000001, -90.195839 ], [ 41.78173799999999, -90.25531 ], [ 41.75646599999999, -90.304886 ], [ 41.722736, -90.326027 ], [ 41.64909, -90.341133 ], [ 41.60279800000001, -90.339348 ], [ 41.586849, -90.348366 ], [ 41.567272, -90.423004 ], [ 41.543578999999994, -90.434967 ], [ 41.527546, -90.454994 ], [ 41.52597, -90.54084 ], [ 41.50958600000001, -90.6007 ], [ 41.46231800000001, -90.658791 ], [ 41.450062, -90.708214 ], [ 41.449820999999986, -90.7799 ], [ 41.44462200000001, -90.844139 ], [ 41.421234, -90.949654 ], [ 41.431084, -91.000694 ], [ 41.423508, -91.027489 ], [ 41.40137899999999, -91.055786 ], [ 41.334895999999986, -91.07328 ], [ 41.267818000000005, -91.102348 ], [ 41.23152200000001, -91.101524 ], [ 41.17625799999999, -91.05632 ], [ 41.16582500000001, -91.018257 ], [ 41.14437100000001, -90.990341 ], [ 41.10435899999999, -90.957787 ], [ 41.07036199999999, -90.954651 ], [ 40.950503999999995, -90.960709 ], [ 40.92392699999999, -90.983276 ], [ 40.87958499999999, -91.04921 ], [ 40.833729000000005, -91.088905 ], [ 40.76154700000001, -91.092751 ], [ 40.70540199999999, -91.119987 ], [ 40.68214800000001, -91.129158 ], [ 40.65631099999999, -91.162498 ], [ 40.64381800000001, -91.214912 ], [ 40.639545, -91.262062 ], [ 40.60343900000001, -91.37561 ], [ 40.572970999999995, -91.411118 ], [ 40.54799299999999, -91.412872 ], [ 40.52849599999999, -91.382103 ], [ 40.50365400000001, -91.374794 ], [ 40.44725, -91.385399 ], [ 40.40298799999999, -91.372757 ], [ 40.392360999999994, -91.385757 ], [ 40.386875, -91.418816 ], [ 40.371902000000006, -91.448593 ], [ 40.309624000000014, -91.486694 ], [ 40.25137699999999, -91.498932 ], [ 40.200458999999995, -91.506546 ], [ 40.134544000000005, -91.516129 ], [ 40.066711, -91.504005 ], [ 40.005753, -91.487289 ], [ 39.94606400000001, -91.447243 ], [ 39.92183700000001, -91.430389 ], [ 39.90182899999999, -91.434052 ], [ 39.885242000000005, -91.450989 ], [ 39.86304899999999, -91.449188 ], [ 39.80377200000001, -91.381714 ], [ 39.76127199999999, -91.373421 ], [ 39.724639999999994, -91.367088 ], [ 39.68591699999999, -91.317665 ], [ 39.600021, -91.203247 ], [ 39.552593, -91.156189 ], [ 39.52892700000001, -91.093613 ], [ 39.473984, -91.064384 ], [ 39.444412, -91.036339 ], [ 39.40058500000001, -90.947891 ], [ 39.35045199999999, -90.850494 ], [ 39.29680300000001, -90.779343 ], [ 39.24780999999999, -90.738083 ], [ 39.22474700000001, -90.732338 ], [ 39.195873000000006, -90.718193 ], [ 39.14421100000001, -90.716736 ], [ 39.09370000000001, -90.690399 ], [ 39.058178, -90.707588 ], [ 39.037791999999996, -90.70607 ], [ 38.93525299999999, -90.668877 ], [ 38.880795000000006, -90.627213 ], [ 38.87132600000001, -90.570328 ], [ 38.89160899999999, -90.530426 ], [ 38.959179000000006, -90.469841 ], [ 38.96233000000001, -90.413071 ], [ 38.92490799999999, -90.31974 ], [ 38.92471699999999, -90.278931 ], [ 38.91450900000001, -90.243927 ], [ 38.85303099999999, -90.132812 ], [ 38.830467, -90.113121 ], [ 38.80051, -90.121727 ], [ 38.785484, -90.135178 ], [ 38.773098000000005, -90.163399 ], [ 38.72396499999999, -90.196571 ], [ 38.70036300000001, -90.20224 ], [ 38.658772, -90.183578 ], [ 38.61027100000001, -90.183708 ], [ 38.562805, -90.240944 ], [ 38.532768000000004, -90.26123 ], [ 38.518688, -90.265785 ], [ 38.427357, -90.301842 ], [ 38.39084600000001, -90.339607 ], [ 38.36533, -90.358688 ], [ 38.32355899999999, -90.369347 ], [ 38.23429899999999, -90.364769 ], [ 38.18871300000001, -90.336716 ], [ 38.16681700000001, -90.289635 ], [ 38.122169000000014, -90.254059 ], [ 38.08890500000001, -90.207527 ], [ 38.05395100000001, -90.134712 ], [ 38.032272000000006, -90.119339 ], [ 37.993206, -90.041924 ], [ 37.969318, -90.010811 ], [ 37.963634, -89.958229 ], [ 37.911884, -89.978912 ], [ 37.878044, -89.937874 ], [ 37.875904000000006, -89.900551 ], [ 37.891875999999996, -89.866814 ], [ 37.905486999999994, -89.861046 ], [ 37.905063999999996, -89.851715 ], [ 37.840992, -89.728447 ], [ 37.804794, -89.691055 ], [ 37.78397, -89.675858 ], [ 37.745453, -89.666458 ], [ 37.706103999999996, -89.581436 ], [ 37.694798000000006, -89.521523 ], [ 37.67984, -89.513374 ], [ 37.650375, -89.51918 ], [ 37.615928999999994, -89.513367 ], [ 37.571957, -89.524971 ], [ 37.491726, -89.494781 ], [ 37.453186, -89.453621 ], [ 37.411018, -89.427574 ], [ 37.355717, -89.435738 ], [ 37.339409, -89.468742 ], [ 37.329441, -89.50058 ], [ 37.304962, -89.513885 ], [ 37.276402000000004, -89.513885 ], [ 37.256001, -89.489594 ], [ 37.253731, -89.465309 ], [ 37.224266, -89.468216 ], [ 37.165318, -89.440521 ], [ 37.137203, -89.423798 ], [ 37.09908299999999, -89.37999 ], [ 37.049212999999995, -89.38295 ], [ 37.009682, -89.310982 ], [ 36.999207, -89.282768 ], [ 37.008686, -89.262001 ], [ 37.027733, -89.264244 ], [ 37.060908999999995, -89.3097 ], [ 37.085384000000005, -89.303291 ], [ 37.091244, -89.284233 ], [ 37.087124, -89.264053 ], [ 37.041732999999994, -89.237679 ], [ 37.02897299999999, -89.210052 ], [ 36.986771000000005, -89.193512 ], [ 36.988113, -89.12986 ], [ 36.99844, -89.150246 ], [ 37.025711, -89.174332 ], [ 37.064235999999994, -89.169548 ], [ 37.093185000000005, -89.146347 ], [ 37.112137000000004, -89.116821 ], [ 37.185860000000005, -89.065033 ], [ 37.22003599999999, -88.993172 ], [ 37.218407, -88.932503 ], [ 37.202194000000006, -88.863289 ], [ 37.152107, -88.746506 ], [ 37.141182, -88.739113 ], [ 37.13540999999999, -88.68837 ], [ 37.109047000000004, -88.61422 ], [ 37.072815000000006, -88.559273 ], [ 37.064769999999996, -88.517273 ], [ 37.06818, -88.4907 ], [ 37.072143999999994, -88.476799 ], [ 37.098670999999996, -88.45047 ], [ 37.156909999999996, -88.422516 ], [ 37.205669, -88.450699 ], [ 37.257782000000006, -88.501427 ], [ 37.296852, -88.511322 ], [ 37.400757, -88.467644 ], [ 37.420292, -88.419853 ], [ 37.40930899999999, -88.359177 ], [ 37.442852, -88.311707 ], [ 37.476273000000006, -88.087883 ], [ 37.51099000000001, -88.071564 ] ] ] } } ], "persistableReferenceCrs": "{"lateBoundCRS":{"wkt":"GEOGCS[\"GCS_North_American_1983\",DATUM[\"D_North_American_1983\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433],AUTHORITY[\"EPSG\",4269]]","ver":"PE_10_3_1","name":"GCS_North_American_1983","authCode":{"auth":"EPSG","code":"4269"},"type":"LBC"},"singleCT":{"wkt":"GEOGTRAN[\"NAD_1983_To_WGS_1984_1\",GEOGCS[\"GCS_North_American_1983\",DATUM[\"D_North_American_1983\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_WGS_1984\",DATUM[\"D_WGS_1984\",SPHEROID[\"WGS_1984\",6378137.0,298.257223563]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"Geocentric_Translation\"],PARAMETER[\"X_Axis_Translation\",0.0],PARAMETER[\"Y_Axis_Translation\",0.0],PARAMETER[\"Z_Axis_Translation\",0.0],AUTHORITY[\"EPSG\",1188]]","ver":"PE_10_3_1","name":"NAD_1983_To_WGS_1984_1","authCode":{"auth":"EPSG","code":"1188"},"type":"ST"},"ver":"PE_10_3_1","name":"NAD83 * DMA-N Am [4269,1188]","authCode":{"auth":"SLB","code":"4269001"},"type":"EBC"}" } }, "relationships": { "projects": { "ids": [ "slb:wing:project-sxs0f1a5219-4640-50af-9f63-c09140d57c4d:" ] } } }, "meta": [ { "kind": "CRS", "name": "GCS_North_American_1983", "persistableReference": "{"lateBoundCRS":{"wkt":"GEOGCS[\"GCS_North_American_1983\",DATUM[\"D_North_American_1983\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433],AUTHORITY[\"EPSG\",4269]]","ver":"PE_10_3_1","name":"GCS_North_American_1983","authCode":{"auth":"EPSG","code":"4269"},"type":"LBC"},"singleCT":{"wkt":"GEOGTRAN[\"NAD_1983_To_WGS_1984_1\",GEOGCS[\"GCS_North_American_1983\",DATUM[\"D_North_American_1983\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_WGS_1984\",DATUM[\"D_WGS_1984\",SPHEROID[\"WGS_1984\",6378137.0,298.257223563]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"Geocentric_Translation\"],PARAMETER[\"X_Axis_Translation\",0.0],PARAMETER[\"Y_Axis_Translation\",0.0],PARAMETER[\"Z_Axis_Translation\",0.0],AUTHORITY[\"EPSG\",1188]]","ver":"PE_10_3_1","name":"NAD_1983_To_WGS_1984_1","authCode":{"auth":"EPSG","code":"1188"},"type":"ST"},"ver":"PE_10_3_1","name":"NAD83 * DMA-N Am [4269,1188]","authCode":{"auth":"SLB","code":"4269001"},"type":"EBC"}", "propertyNames": [ "SpatialLocation.AsIngestedCoordinates" ] } ], "id": "opendes:teststates:relationships-aWxsaW5vaXM", "version": 1631682054027397, "kind": "opendes:relationships:teststates:1.0.0", "acl": { "viewers": [ "data.default.viewers@opendes.enterprisedata.cloud.slb-ds.com" ], "owners": [ "data.default.viewers@opendes.enterprisedata.cloud.slb-ds.com" ] }, "legal": { "legaltags": [ "opendes-public-usa-dataset-7643990" ], "otherRelevantDataCountries": [ "US" ], "status": "compliant" }, "createUser": "39916b94-71a9-409e-856e-0f29558fa908", "createTime": "2021-09-15T05:00:55.838Z" }
Please let me know if you need any other information.
Thanks and Regards, Monalisa Srivastava
Schlumberger-Private From: Sanjeev Pellikoduku SPellikoduku@slb.com Sent: Wednesday, September 15, 2021 2:00 AM To: Monalisa Srivastava MSrivastava4@slb.com Cc: Neelesh Thakur NThakur4@slb.com; Bhakti Ashwin Thakkar BThakkar@slb.com; Nitin Jain NJain5@slb.com Subject: RE: Issue with search
Hi Monalisa,
The records were ingested using multiple kinds "kind": "opendes:staticrela:states:3.0.0" and "kind": "opendes:staticrela:states:4.0.0".
"kind": "opendes:staticrela:states:3.0.0" "totalCount": 40 "kind": "opendes:staticrela:states:4.0.0" "totalCount": 49
Below 9 records ingested with a kind “opendes:staticrela:states:4.0.0”.
opendes:states:staticrela-aWxsaW5vaXMxNw opendes:states:staticrela-bWlzc291cmkyOQ opendes:states:staticrela-dGVubmVzc2VlNDc opendes:states:staticrela-bWlzc2lzc2lwcGkyOA opendes:states:staticrela-YXJrYW5zYXM1 opendes:states:staticrela-bG91aXNpYW5hMjI opendes:states:staticrela-bWljaGlnYW4yNg opendes:states:staticrela-d2lzY29uc2luNTU opendes:states:staticrela-bWlubmVzb3RhMjc
Please try this query { "kind": "opendes:staticrela:states:4.0.0", "returnedFields": [ "id", "index" ] }
Regards. Sanjeev
Schlumberger-Private From: Monalisa Srivastava MSrivastava4@slb.com Sent: Tuesday, September 14, 2021 6:45 AM To: Sanjeev Pellikoduku SPellikoduku@slb.com Cc: Neelesh Thakur NThakur4@slb.com; Bhakti Ashwin Thakkar BThakkar@slb.com Subject: Issue with search
Hi Sanjeev,
Need you help to validate an issue we are facing with the search service, below are the details – Env QA
The shapefile ingestor ingested 49 records :
opendes:states:staticrela-aWxsaW5vaXMxNw opendes:states:staticrela-bWlzc291cmkyOQ opendes:states:staticrela-dGVubmVzc2VlNDc opendes:states:staticrela-bWlzc2lzc2lwcGkyOA opendes:states:staticrela-YXJrYW5zYXM1 opendes:states:staticrela-bG91aXNpYW5hMjI opendes:states:staticrela-bWljaGlnYW4yNg opendes:states:staticrela-d2lzY29uc2luNTU opendes:states:staticrela-bWlubmVzb3RhMjc opendes:states:staticrela-ZGlzdHJpY3Qgb2YgY29sdW1iaWExMQ opendes:states:staticrela-ZGVsYXdhcmUxMA opendes:states:staticrela-d2VzdCB2aXJnaW5pYTU0 opendes:states:staticrela-bWFyeWxhbmQyNA opendes:states:staticrela-Y29sb3JhZG84 opendes:states:staticrela-a2VudHVja3kyMQ opendes:states:staticrela-a2Fuc2FzMjA opendes:states:staticrela-dmlyZ2luaWE1MQ opendes:states:staticrela-YXJpem9uYTQ opendes:states:staticrela-b2tsYWhvbWE0MA opendes:states:staticrela-bm9ydGggY2Fyb2xpbmEzNw opendes:states:staticrela-dGV4YXM0OA opendes:states:staticrela-bmV3IG1leGljbzM1 opendes:states:staticrela-YWxhYmFtYTE opendes:states:staticrela-Z2VvcmdpYTEz opendes:states:staticrela-c291dGggY2Fyb2xpbmE0NQ opendes:states:staticrela-ZmxvcmlkYTEy opendes:states:staticrela-bW9udGFuYTMw opendes:states:staticrela-bWFpbmUyMw opendes:states:staticrela-bm9ydGggZGFrb3RhMzg opendes:states:staticrela-c291dGggZGFrb3RhNDY opendes:states:staticrela-d3lvbWluZzU2 opendes:states:staticrela-aWRhaG8xNg opendes:states:staticrela-dmVybW9udDUw opendes:states:staticrela-b3JlZ29uNDE opendes:states:staticrela-bmV3IGhhbXBzaGlyZTMz opendes:states:staticrela-aW93YTE5 opendes:states:staticrela-bWFzc2FjaHVzZXR0czI1 opendes:states:staticrela-bmVicmFza2EzMQ opendes:states:staticrela-bmV3IHlvcmszNg opendes:states:staticrela-cGVubnN5bHZhbmlhNDI opendes:states:staticrela-Y29ubmVjdGljdXQ5 opendes:states:staticrela-cmhvZGUgaXNsYW5kNDQ opendes:states:staticrela-bmV3IGplcnNleTM0 opendes:states:staticrela-aW5kaWFuYTE4 opendes:states:staticrela-bmV2YWRhMzI opendes:states:staticrela-dXRhaDQ5 opendes:states:staticrela-Y2FsaWZvcm5pYTY opendes:states:staticrela-b2hpbzM5 opendes:states:staticrela-d2FzaGluZ3RvbjUz
However the search query gives me only 40 records, I even used the index getting 200 however the records count and ids are 40 :
{ "kind": "opendes:staticrela:states:3.0.0", "returnedFields": [ "id", "index" ], "limit": 100 }
Response :
{ "cursor": "75E5D618FA7C168226980DA08027ECAB", "results": [ { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bmV3IHlvcmszNg" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bm9ydGggY2Fyb2xpbmEzNw" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-aW5kaWFuYTE4" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-b2tsYWhvbWE0MA" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-YWxhYmFtYTE" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-aWRhaG8xNg" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bmV3IGhhbXBzaGlyZTMz" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-b3JlZ29uNDE" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bmV2YWRhMzI" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-d2VzdCB2aXJnaW5pYTU0" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-dmlyZ2luaWE1MQ" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bm9ydGggZGFrb3RhMzg" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-ZGlzdHJpY3Qgb2YgY29sdW1iaWExMQ" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-c291dGggY2Fyb2xpbmE0NQ" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-d2FzaGluZ3RvbjUz" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-YXJpem9uYTQ" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-Y29ubmVjdGljdXQ5" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-b2hpbzM5" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bWFzc2FjaHVzZXR0czI1" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-a2Fuc2FzMjA" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bmV3IG1leGljbzM1" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-cmhvZGUgaXNsYW5kNDQ" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-Y29sb3JhZG84" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-Y2FsaWZvcm5pYTY" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bWFpbmUyMw" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-a2VudHVja3kyMQ" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-aW93YTE5" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bmV3IGplcnNleTM0" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-dmVybW9udDUw" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-Z2VvcmdpYTEz" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-d3lvbWluZzU2" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bW9udGFuYTMw" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-dXRhaDQ5" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bmVicmFza2EzMQ" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-ZmxvcmlkYTEy" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-bWFyeWxhbmQyNA" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-ZGVsYXdhcmUxMA" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-c291dGggZGFrb3RhNDY" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-dGV4YXM0OA" }, { "index": { "trace": [], "statusCode": 200, "lastUpdateTime": "2021-09-14T08:14:52.956Z" }, "id": "opendes:states:staticrela-cGVubnN5bHZhbmlhNDI" } ], "totalCount": 40 }
Missing 9 records :
opendes:states:staticrela-aWxsaW5vaXMxNw opendes:states:staticrela-bWlzc291cmkyOQ opendes:states:staticrela-dGVubmVzc2VlNDc opendes:states:staticrela-bWlzc2lzc2lwcGkyOA opendes:states:staticrela-YXJrYW5zYXM1 opendes:states:staticrela-bG91aXNpYW5hMjI opendes:states:staticrela-bWljaGlnYW4yNg opendes:states:staticrela-d2lzY29uc2luNTU opendes:states:staticrela-bWlubmVzb3RhMjc
As discussed today, based on further investigation the Elesticsearch is not able to index the geo_shape defined in the 9 missing records.
reason=Unable to Tessellate shape. It’s a bug in Lucene tessellator. I think it’s fixed in the latest version of the Elastic search.
All the below records have several more coordinates with the type “AnyCrsPolygon” and “AnyCrsMultiPolygon” than the other records.
opendes:teststates:relationships-aWxsaW5vaXM opendes:teststates:relationships-bWlzc291cmk opendes:teststates:relationships-dGVubmVzc2Vl opendes:teststates:relationships-bWlzc2lzc2lwcGk opendes:teststates:relationships-YXJrYW5zYXM opendes:teststates:relationships-bG91aXNpYW5h opendes:teststates:relationships-bWljaGlnYW4 opendes:teststates:relationships-d2lzY29uc2lu opendes:teststates:relationships-bWlubmVzb3Rh