Advanced Resource Connector

Advanced Resource Connector (ARC) is a grid computing middleware introduced by NorduGrid. It provides a common interface for submission of computational tasks to different distributed computing systems and thus can enable grid infrastructures of varying size and complexity. The set of services and utilities providing the interface is known as ARC Computing Element (ARC-CE).[2] ARC-CE functionality includes data staging and caching, developed in order to support data-intensive distributed computing.[3] ARC is an open source software distributed under the Apache License 2.0.[1]

ARC
ARC logo and monitor screenshot
Developer(s)NorduGrid, NeIC, EU projects
Initial releaseApril 13, 2004 (2004-04-13)
Stable release
6.6 / 27 April 2020
Written inC++
Operating systemLinux, Microsoft Windows, Mac OS X
Available inEnglish, Russian, Swedish
TypeGrid computing
LicenseApache License 2.0[1]
Websitewww.nordugrid.org

History

ARC appeared (and is still often referred to) as the NorduGrid middleware, originally proposed as an architecture on top of the Globus Toolkit[4] optimized for the needs of High-Energy Physics computing for the Large Hadron Collider experiments.[5] First deployment of ARC at the NorduGrid testbed took place in summer 2002, and by 2003 it was used to support complex computations.[6]

The first stable release of ARC (version 0.4) came out in April 2004 under the GNU General Public License.[7] The name "Advanced Resource Connector" was introduced for this release to distinguish the middleware from the infrastructure. In the same year, the Swedish national Grid project Swegrid became the first large cross-discipline infrastructure to be based on ARC.[8]

In 2005, NorduGrid was formally established as a collaboration to support and coordinate ARC development.[9] In 2006 two closely related projects were launched: the Nordic Data Grid Facility, deploying a pan-Nordic e-Science infrastructure based on ARC, and KnowARC, focused on transforming ARC into a next generation Grid middleware.

ARC v0.6 was released in May 2007, becoming the second stable release.[10] Its key feature was introduction of the client library enabling easy development of higher-level applications. It was also the first ARC release making use of open standards, as it included support for JSDL. Later that year, the first technology preview of the next generation ARC middleware was made available, though was not distributed with ARC itself.[11] The new approach involved switching to a Web service based architecture, and in general a very substantial re-factorisation of the core code.

In 2008, the NorduGrid consortium adopted the Apache License for all ARC components.

The last stable release in the 0-line was ARC v0.8, shipped in September 2009.[12] It eventually included a preview version of the new execution service - the A-REX' - and several other components, like Chelonia, ISIS, Charon' and the arcjobtool GUI.

In parallel to ARC v0.8, the EU KnowARC project released in November 2009 the conceptual ARC NOX suite, which was a complete Grid solution, fully based on Web service technologies.[13] The name NOX actually indicates the release date: November of the Year of the Ox.

In May 2011, NorduGrid released ARC v11.05 (adopting Ubuntu versioning scheme this time). This release marked the complete transition from the old execution service to A-REX and accompanying services. For backwards compatibility with the existing infrastructures, old interfaces for the execution service and the information system were retained.

ARC 6 was released in May 2019[14] and while having same interfaces it features a completely redesigned configuration and a new management tool.

Components
  • Hosting environment:
  • Execution services:
    • Grid Manager - server providing computing capability
    • A-REX - next generation of Grid Manager, WS-based
  • Information services:
    • Classic information server - local service information publishing
    • Classic information index - distributed information indexing
  • Accounting:
    • JURA - job record publisher for A-REX
  • Clients:
    • arc* job and data CLI - next generation command-line interface for job and data management, interoperable with other middlewares like gLite and UNICORE
    • arcproxy - universal Grid proxy generation utility
    • Grid Monitor - graphical interface to the classic information services
  • Libraries and API:
    • libarcclient - job management and information
    • libarcdata2 - data management
    • HED security API

Availability

ARC is free software available from the NorduGrid public repository, both as binary packages for a variety of Linux systems and source. Source code is also openly available from the NorduGrid SVN repository.

Development

The open source development of the ARC middleware is coordinated by the NorduGrid collaboration. Contributions to the software, documentation and dissemination activities are coming from the community and from various projects, such as the EU KnowARC and EMI projects, NDGF, NeIC and various national infrastructure and research projects.

Versioning


Between 2011 and 2018 ARC used an Ubuntu-like versioning schema for bundled releases consisting of individual components. Individual components have own versioning, corresponding to code tags.[15] Version of the core ARC packages is often used instead of the formal release number in everyday communication.

Starting with ARC6 (2019) version number of the release coincides with that of the tag.

Standards and interoperability

ARC implements several Open Grid Forum standards, in particular, JSDL, Glue2, BES, UR/RUS and StAR.[16]

ARC in various projects and initiatives

European Middleware Initiative

In 2010-2013, several key ARC components - most notably, HED, A-REX, clients and libraries - were included in the European Middleware Initiative (EMI) software stack. Through EMI, ARC became a part of the Unified Middleware Distribution (UMD) of the European Grid Infrastructure (EGI).

Nordic DataGrid Facility and NeIC

ARC is the basis of the computing infrastructure of the Nordic Data Grid Facility (NDGF). In 2006-2010 NDGF actively contributed to ARC development, and since 2010 provides ARC deployment expertise within EGI. Since 2012, NDGF became a part of the Nordic e-Infrastructure Collaboration.[17]

KnowARC project

Grid-enabled Know-how Sharing Technology Based on ARC Services and Open Standards (KnowARC) was a Sixth Framework Programme Specific Targeted Research Project, funded under Priority IST-2005-2.5.4 "Advanced Grid Technologies, Systems and Services" from June 2006 to November 2009.[18][19] In many ways it was the project that shaped ARC. The main goal was to make ARC based on open community standards, and among the key results was creation of the standardized Hosting Environment for ARC services (HED).

Apart from its main aim of further developing ARC,[20] it contributed to the development of standards,[21] and increased Grid and ARC usage in medicine and bioinformatics.[22][23]

In July 2009, KnowARC announced it contributed to the integration of Grid technologies into official Linux repositories by adding Globus Toolkit components into Fedora and Debian repositories.[24]

gollark: OH BEE.
gollark: So far.
gollark: Oh, that was fine.
gollark: We could probably copy things in the right size as emotes.
gollark: Too big.

See also

References
  1. http://download.nordugrid.org/
  2. "ARC Computing Element System Administrator Guide" (PDF). NorduGrid. 25 June 2015. Retrieved 26 June 2015.
  3. Ellert, Mattias; et al. (February 2007). "Advanced Resource Connector middleware for lightweight computational Grids". Future Generation Computer Systems. 23 (2): 219–240. doi:10.1016/j.future.2006.05.008.
  4. Ellert, Mattias; Konstantinov, Aleksandr; Kónya, Balázs; Smirnova, Oxana; Wäänänen, Anders (2003). "The NorduGrid project: using Globus toolkit for building GRID infrastructure". Nuclear Instruments and Methods in Physics Research A. 502 (2–3): 407–410. Bibcode:2003NIMPA.502..407E. doi:10.1016/S0168-9002(03)00453-4.
  5. Wäänänen, Anders; Ellert, Mattias; Konstantinov, Aleksandr; Kónya, Balázs (2002). "An Overview of an Architecture Proposal for a High Energy Physics Grid". In Fagerholm, Juha; Haataja, Juha; Järvinen, Jari; Lyly, Mikko; Råback, Peter; Savolainen, Ville (eds.). Lecture Notes in Computer Science. 2367. Springer. pp. 76–86. doi:10.1007/3-540-48051-X_9. ISBN 978-3-540-43786-4.
  6. Eerola, Paula; et al. (2003). "Atlas Data-Challenge 1 on NorduGrid". Proceedings of 2003 Conference for Computing in High Energy and Nuclear Physics. arXiv:physics/0306013. Bibcode:2003physics...6013E.
  7. ARC 0.4 Release Notes
  8. "SweGrid gets set for future challenges". CERN Courier. 2004.
  9. NorduGrid Web site
  10. ARC 0.6 Release Notes
  11. "KnowARC report D5.1-2_07" (PDF). Archived from the original (PDF) on 2010-11-08. Retrieved 2009-08-22.
  12. ARC 0.8 Release Notes
  13. ARC NOX Release Notes
  14. ARC 6 Release Notes
  15. ARC releases table
  16. W. Qiang (31 October 2012). Transparent use of open standards in the EMI component ecosystem (Report). CERN.
  17. NeIC Web site
  18. KnowARC fact-sheet, EU IST database
  19. Hämmerle, Hannelore; Crémel, Nicole (November 2006). "KnowARC project gets going". CERN Courier. Geneva, Switzerland. 46 (11): 12.
  20. Smirnova, Oxana; et al. (2009). "ARC middleware:evolution towards standards-based interoperability" (PDF). Proceedings of the 17th International Conference on Computing in High Energy and Nuclear Physics.
  21. Field, Laurence; Andreozzi, Sergio; Kónya, Balázs (2008). Grid Information System Interoperability: The Need For A Common Information Model. Proceedings of the IEEE Fourth International Conference on EScience. pp. 501–507. doi:10.1109/eScience.2008.159. ISBN 978-1-4244-3380-3.
  22. Zhou, Xin; et al. (2009). "An Easy Setup for Parallel Medical Image Processing: Using Taverna and ARC". Studies in Health Technology and Informatics. 147 (Healthgrid Research, Innovation and Business Case): 41–50. doi:10.3233/978-1-60750-027-8-41. PMID 19593043.
  23. Krabbenhöft, Hajo; Möller, Steffen; Bayer, Daniel (2008). "Integrating ARC grid middleware with Taverna workflows". Bioinformatics. 24 (9): 1221–1222. doi:10.1093/bioinformatics/btn095. PMID 18353787.
  24. "KnowARC Project Brings Grids to Debian". HPC Wire. July 9, 2009. Archived from the original on September 5, 2009.

Further reading
  • Qiang, Weizhong; Konstantinov, Aleksandr (June 2009). "Towards cross-middleware authentication and single sign-on for ARC Grid middleware". Computer Science - Research and Development. 23 (3–4): 267–274. doi:10.1007/s00450-009-0084-6.
  • Anderlik, Csaba; Gregersen, Anders Rhod; Kleist, Josva; Peters, Anders; Saiz, Pablo (2008). "ALICE-ARC Integration". Journal of Physics: Conference Series. 119 (62006): 6pp. doi:10.1088/1742-6596/119/6/062006.
  • Jensen, Henrik Thostrup; Kleist, Josva; Leth, Jesper Ryge (2005). A Framework for Job Management in the NorduGrid ARC Middleware. Lecture Notes in Computer Science. 3470. Berlin/Heidelberg: Springer. pp. 861–871. doi:10.1007/11508380_88. ISBN 978-3-540-26918-2.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.