Äîêóìåíò âçÿò èç êýøà ïîèñêîâîé ìàøèíû. Àäðåñ îðèãèíàëüíîãî äîêóìåíòà : http://sinp.msu.ru/en/system/files/preprints/pp-882.pdf Äàòà èçìåíåíèÿ: Fri Feb 28 00:51:18 2014 Äàòà èíäåêñèðîâàíèÿ: Fri Feb 28 00:51:18 2014 Êîäèðîâêà:

..
- ..

.. , .. , .. , .. , .. , .., .. RESTFUL--

2012-4/882

, 2012 .



001.89:004 .. , .. , .. , .. , .. , .., .. e-mail: kryukov@theory.sinp.msu.ru RESTFUL-- 2012-4/882 RESTful-- , : , , , UNIDATA Common Data Model, , , , , , . . ( 07.514.11.4022 20.09.2011). A.P.Kryukov, M.N.Zhizhin, FA.A.Poyda, D.Yu.Mishin, D.P.Medvedev, A.P.Demichev, D.S.Kokovin DEVELOPMENT OF RESTFUL-WEB SERVICES FOR REMOTE ACCESS TO DATA STORAGE RESOURCES FOR DISTRIBUTED SYSTEMS IN EARTH SCIENCES Preprint MSU SINP 2012-4/882 Abstract RESTful-web services for remote access to data storage resources for distributed systems in the Earth sciences are presented. They possess the following features: operation support with meta data and large volumes of data including search and automatic generation of requests to data on the basis of meta data, support of the general data model compatible to UNIDATA Common Data Model and allowing to integrate heterogeneous resources in the field of Earth sciences, support of the generalized language of the requests allowing to create the universal requests to several heterogeneous data sources, horizontal scalability, stream asynchronous request of the data allowing to avoid timeout errors and memory overflow on a remote server.
© .. , .. , .. , .. , .. , .., .. © , , http://www.sinp.msu.ru




, , : · . , , , , . , , . . , - , . , ( , ). · . ­ - . , (, , NCEP/NCAR [1], , ). , . · . , , . , , , , . , . · . - . , , , , . , . : (.. ). ( ), . : o , , , , . . o .


. RESTful-- , : , , , UNIDATA Common Data Model [2], , , , , , . , : , , , - . , . « 2007-2013 » «2011-1.4-514-015» « RESTful-- - » « RESTful-- - , » ( «2011-1.4514-015-081»).
o

1.
, . UNIDATA Common Data Model (CDM) [2]. CDM , ( 1) - , : 1. (data access layer) . 2. . , . 3. , , , , .., .














1. UNIDATA Common Data Model

CDM 2.

2. CDM


Dataset ( ) ( NetCDF, HDF5, GRIB ), - OPeNDAP [3], , NetCDF [4]. Group () ­ , , , . Variable () ­ , , . Dimension () ­ . . Attribute () ­ , , , . Structure () ­ , . Sequence () ­ ( , . , , ). Array () ­ , . ArrayStructure ( ) ­ , . UNIDATA CDM 3.

3. UNIDATA CDM

Variable () , . Coordinate System ( ) . CoordinateAxis ( ) ­ AxysType. CoordinateTransform ­ .


AxisType - , . , , , , , .. . , - . , UNIDATA CDM.
ANY // any type NONE // no type GRID RADIAL SWATH IMAGE ANY_POINT // any of the following POINT PROFILE SECTION STATION STATION_PROFILE TRAJECTORY

RESTful , , . ( 4), NetCDF-3 [5] UNIDATA CDM. UNIDATA CDM.

4. ,

, UNIDATA CDM , Unidata CDM. , .


2.
, , , , . : · « »; · , ; · , · . UNIDATA CDM (. ). , . ( 1).
1.

command



dataset format datefrom

@theme , ,

get ( .) ­ describe ­ XML param[.vlevel[.#]]@theme[.source] geom_z@Geom geom_y.Geom_yr@Geom.yr SkinTemperature.Surface@Weather xml, json, ncml, csv, csv_noheader, jpeg, jpg, png, tif, tiff yyyy-mm-ddThh:mm:ssUTC

dateto

yyyy-mm-ddThh:mm:ssUTC - Point: (56.0,38.0)

location

- ROI (Region of Interest): (., ., .., .) true ­

async asyncorder


5 . , 100 56,38 (, ), :
http://93.180.14.101:18080/restful/servlet/GetData2?format=csv&datefrom=201103-01T00:00:00UTC&dateto=2011-0303T00:00:00UTC&dataset=Temperature.IsobaricLevel.100@Weather&location=(56,38)

5.

3.

, . ( 6) , . : · · · · · · · -


·




- WS - Outer search





Store Plugin

Store Plugin

Resource Plugin

WS ­ API

WS ­ API

WS ­ API

User DB

XML





6. ,

, . , , , ( 7).


( ) Wiki























7.

XML . , .

XML
, , XML . . , (, , ..). XML , ( ). ( , ) . , , , , . :


·

; ; ( ); , ( ); ; , .

· · · · ·

XML , . , ( , ..).


, ( 8), XML (, , , ..).







Plugin







8.

. , , XML . (, , , ..) .


. , ( ), , , . , , , . .


«», . , , : , , , .. , ( 9). , ( , ). , , , , . «». () , . , . , . , «». , , , , . , , , , .


XML





Old on versi



()

( ) ...





9.


. . : , , , . : . . : (, ). , , . : -, , -. , , - -, , . - REST. - - , . , . - : , . : , .


, . , XML , .


( 10). ­ . , , . , , , , . ­ , . , THREDDS OPeNDAP [3] , . , , , . , API. , , .


(OE XML )




(web-page)


DataQuer yAction (MVC class )


Service plugin ( class )









, . .

10.

, , . , . , (), . , , .

4.
, , , , , . , « latlon» «location» . , , , . , ,


. , . , , ( «OE») () . , : 1. , . 2. , . , , . . 3. , . «». 11.

XSLT

( XML OE )



- HT ML *.class


-


(: , )

11.

- , XSLT . , , , , . RESTful . : 1. OE. 2.




public org.jdom.Element process(Map params, String paramsString, String basketObjectId){ // ... // ... . // : FileStoreSave.storeFile(inp, basketObjectId, "XML file", null, basketObjectId, "fileAction"); // ... Element returnElm = new Element("result"); returnElm.addContent( (new Element("status")).setText("0")); returnElm.addContent( (new Element("error")).setText("")); return returnElm; }

, . . , , , :

, «1» 3. . " vo_store.conf" :
< >.pluginClass=< >

. , «voPlugin» ­ .
voPlugin

Java Struts -, XML XSLT HTML . - Tomcat.

5.
­ RESTful , , , . . , : · ; · (, ); · ; · .


, UNIDATA CDM, , . , . CDM , : XML, JSON [6-7], NetCDF , NcML. , , , , (. 2). , . , , ( .. ). RESTful , web- . , , . , ( throttling ). RESTful . - RESTful . , , . « stateful» , , . , . , , . RESTful : REST , . RESTful , HTTP , . . . "" . " " , . . . «stateful» , . , . : . - . , , , .


. RESTful . , "processing". , "error" . , "order not found". , . , . " EOF". 12. , . , .

12.

6. -
·

- ( ) : . ( ) XML- OE, . , .


. (, - ). · , ( ) . , -.
·

7.
7.1.
NCEP/NCAR NCEP/NCAR [1] 1948 . . , . NCEP/NCAR : 2.5 2.5 (73 , 144 ) T62 (94 , 192 ). 4 . (: - ), . 8 17. 13 NCEP/NCAR. , , .


13. NCEP/NCAR

NCEP/NCAR NetCDF . 74 , 5000 NetCDF 30 ­ 500 . NCEP/NCAR 500 . DMSP OLS- DMSP ( F10 ­ F18). TIFF 30 [8]. -180 180 -65 75 . , , .. . 1992 2010 . , , , 1994 F10 F12. 2003 F15 14.

14. , DMSP F15, 2003


, : .

7.2. -
NWS. : 1. , «/NWS weather» ( 15).

15. -

2. , , , , ( «») ( 16).

16.

3. «», . ( 17).


17.


RESTful-- : , , , UNIDATA Common Data Model, , , , , , . ( 07.514.11.4022 20.09.2011).


[1]. Kalnay et al.,The NCEP/NCAR 40-year reanalysis project, Bull. Amer. Meteor. Soc., 77, 437-470, 1996. [2]. Unidata Common Data Model. : http://www.unidata.ucar.edu/software/netcdf-java/CDM/ [3]. OPeNDAP. , URL: http://opendap.org/ [4]. Network Common Data Form (NetCDF). : http://www.unidata.ucar.edu/software/netcdf/ [5]. NetCDF-3 NetCDF-4 . . URL: http://www.unidata.ucar.edu/projects/THREDDS/GALEON/Reports/CDMISO/CDM-ISO-DataModels.htm [6]. D. Crockford "The application/json Media Type for JavaScript Object Notation (JSON)" Technical report, IETF Network Working Group, RFC4627 (2006) [7]. K. Zyp "A JSON Media Type for Describing the Structure and Meaning of JSON Documents" Technical report, IETF Network Working Group (2010)


[8]. DMSP-OLS. (NGDC NOAA). : http://www.ngdc.noaa.gov/dmsp/downloadV4composites.html




RESTFUL--

2012-4/882

20.07.2012