Univa Grid Engine

Univa Grid Engine is a batch-queuing system, forked from Sun Grid Engine (SGE).[2][3] The software schedules resources in a data center applying user-configurable policies to help improve resource sharing and throughput by maximizing resource utilization. The product can be deployed to run on-premises, using IaaS cloud computing or in a hybrid cloud environment.[4]

Univa Grid Engine
Developer(s)Univa
Stable release
8.6.12 / 15 April 2020 (2020-04-15)
Operating systemCross-platform
TypeGrid computing Supercomputing
LicenseProprietary commercial software[1]
Websitewww.univa.com/products

History

The roots of Grid Engine as a commercial product date back to 1993 (under the names CODINE and later, in a variation of the product, GRD). A more comprehensive genealogy of the product is described in Sun Grid Engine. Grid Engine was first distributed by Genias Software and from 1999, after a company merger, by Gridware, Inc. In 2000, Sun Microsystems acquired Gridware.[5] Sun renamed CODINE/GRD as Sun Grid Engine later that year, and released it as open-source in 2001.[6]

In 2010, Oracle Corporation acquired Sun and subsequently renamed SGE to Oracle Grid Engine. Oracle Grid Engine (6.2u6) moved to a closed-source model providing binaries with the distribution but no source code. As a result, the project's open-source repository no longer reflected changes made by Oracle and users were prevented from contributing code changes. In response to this, the Grid Engine community started the Open Grid Scheduler and the Son of Grid Engine projects to continue to develop and maintain a free implementation of Grid Engine.[7][8][9]

On January 18, 2011, Univa announced that it had hired the principal engineers from the Sun Grid Engine team.[10] Univa Grid Engine development is led by CTO Fritz Ferstl, who founded the Grid Engine project and ran the business within Sun/Oracle for the past 10 years.[11]

On October 22, 2013 Univa announced that it had acquired Oracle Grid Engine assets and intellectual property making it the sole commercial provider of Grid Engine software.[12]

Between 2011 and 2013 Univa added new capabilities to Univa Grid Engine including Univa Unisight[13][14], and Univa License Orchestrator.[15][16]

Univa Unisight provided new reporting and analytics capabilities related to of Univa Grid Engine workloads and infrastructure. Univa License Orchestrator extended Univa Grid Engine scheduling policies to support allocation and optimization of commercial software licenses, an important capability in Electronic Design Automation (EDA) and other industries.

On June 24, 2018 Univa announced massive scalability operating a single cluster with over 1 million cores on AWS.[17]

Releases

Univa Grid Engine 8.0 was Univa’s first commercial release of Grid Engine, offered on April 12, 2011.[18] It was forked from SGE 6.2u5, the last open source release [19]. It added improved third party application integration, license and policy management, enhanced support for software and hardware platforms, and cloud management tools.

Univa Grid Engine 8.0.1 was released on October 4, 2011. It adds improved support for multi-core hardware, integration with NVIDIA GPUs, new job submission verifier extensions, and additional bug fixes.[20]

Univa Grid Engine 8.1.0 was announced on May 2, 2011 and improvements and bug fixes were made in a series of 8.1.x releases through 8.1.3 announced on Nov 15, 2012[21]. The 8.1.x releases delivered important new functionality such as Job Classes, PostgreSQL Spooling, Resource Maps, and improvements to Share Tree Policies. The releases also introduced a new Fair Urgency Policy, deterministic Wildcard PE Selection, improved Diagnostics, pre-configured MPI integrations, an improved Apache Hadoop Integration as well as many bug fixes and performance improvements.

Univa Grid Engine 8.1.6 was announced on Oct 14, 2013. This update included improvements to the Univa Grid Engine scheduler aimed at larger clusters and Qmaster stability and scalability improvements.[22]

Univa Grid Engine 8.2.0 was released on Sept 02, 2014. Univa Grid Engine version 8.2.0 was the first release to provide native support for Microsoft® Windows® environments.[23]

Univa Grid Engine 8.3.0 was released on June 22, 2015. The new Preemption feature in Univa Grid Engine 8.3.0 allowed users to set priorities on different work so that, if a higher priority application needed to use resources allocated to a lower priority application, the lower priority application would be effectively be “paused”—not lost—and work would automatically resume once the higher priority application completed. Among a handful of other new features added to Grid Engine 8.3.0 to improve overall reliability and efficiency was a new Run Time Modification of Resources feature. The Run Time Modification of Resources feature enabled cluster administrators to make configuration changes “on the fly” improving cluster availability and improving overall efficiency.[24]

Univa Grid Engine 8.3.1 was released on August 28, 2015. This release contained additional fixes and enhancements identified since the release of 8.3.0.[25]

Univa Grid Engine 8.4.0 was released on May 31, 2016. This release supports Docker containers containers and automatically dispatched and ran jobs within a user specified Docker Image.[26][27]

Univa Grid Engine 8.5.0 was released on March 7, 2017. This release of Univa Grid Engine provided on average ~2x faster scheduling than open source Grid Engine 6.2u5. Univa Grid Engine 8.5.0 also delivered significant improvements to Docker support including mobility of GPU apps within a cluster.[28]

Univa Grid Engine 8.6.0 was released on July 17, 2018. This release added support for NVIDIA Docker 2.0 providing more flexibility when running Docker containers in a Univa Grid Engine environment.[29]

Univa Grid Engine 8.6.1 was released on August 8, 2018, providing improved control over GPU devices, and new affinity features, allowing jobs to gravitate towards, or away from, certain compute nodes.[30]

Univa Grid Engine 8.6.2 was released on August 16, 2018. This release improved Univa Grid Engine performance and scalability in several key areas including network communications, job submission, memory allocation, and scheduler optimizations. This update also improved Univa Grid Engine job dispatch information.[31]

Univa Grid Engine 8.6.3 was released on September 27, 2018. This update introduced bulk configuration changes for Univa Grid Engine hosts. Bulk configuration changes perform operations on many hosts simultaneously, making it easier to manage large Univa Grid Engine clusters.[32]

Univa Grid Engine 8.6.4 was released on November 23, 2018, providing new core binding strategies making it easier to specify how jobs are placed on nodes and cores while also providing more flexibility. A new affinity-based job placement policy was included in this update. Jobs submitted using affinity can be packed close together (“positive affinity”) or spread across the cluster (“negative affinity”) based on resources requested. New Univa Grid Engine Resource Maps syntax provides more granular control over application access to host devices such as NVIDIA GPUs. This allows jobs to request GPUs and ensure that GPUs are exclusively assigned to the specific job. Univa Grid Engine was also enhanced to directly communicate with NVIDIA Data Center GPU Manager (DCGM) in this release to collect GPU metrics for scheduling and accounting.[33]

Univa Grid Engine 8.6.5 was released on May 6, 2019. Key new features were:

  • Support for IBM Power 9 on Linux
  • Improvements to Docker support on Univa Grid Engine
  • Very large cluster hostname management where the IP address for each host is contained in the hostname
  • Integration with Linux Out of Memory (OOM) Notification API ensuring that Univa Grid Engine is automatically notified of jobs that are terminated by the Linux kernel
  • Improved responsiveness in heavily loaded cluster for Grid Engine administrator commands
  • Performance improvements to Univa Grid Engine automatic job rescheduling
  • Thread deadlock detection for Univa Grid Engine Qmaster
  • Updated support for NVIDIA DCGM versions up to 1.6.3
  • Ability to specify GPU/CPU affinity as hard or soft requests

Univa Grid Engine 8.6.7 was released on August 5, 2019offering Red Hat Enterprise Linux 8 (RHEL8) support and NVIDIA GPU DCGM job usage features.[34]

Univa Grid Engine 8.6.8 was released on December 12, 2019, providing new parameters to fine-tine scheduling wildcard requests, support for Linux mount namespaces and GPU usage reporting.[35]

Univa Grid Engine 8.6.9 was released on February 10, 2020, providing enhancements to the qconf command and improved information and messaging collection.[36]

Univa Grid Engine 8.6.11 was released on March 17, 2020 delivering improved Docker compatibility, job reporting and monitoring and increased support for the latest version of DCGM.[37]

gollark: That's a bad reason.
gollark: In most cases.
gollark: And thanks to package manager technology, you don't have to spend ages finding trustworthy sources for the EXE.
gollark: You can happily use AppImages and whatever on Linux, you know.
gollark: Single click after tracking down the right EXE, you mean?

See also

References
  1. Univa Support and Term Software license
  2. Gentzsch, Wolfgang (2011-01-18). "Grid Engine Finds Safe Harbor at Univa". HPC in the Cloud. Retrieved April 17, 2011.
  3. Morgan, Timothy Prickett (2011-01-18). "Univa forks Oracle's Sun Grid Engine". The Register. Retrieved April 17, 2011.
  4. Fritz Ferstl (2020-02-12). "Grid Engine in the Age of Cloud". Univa.
  5. "Sun Microsystem to acquire GRIDWARE". hpcwire. 2000-07-28.
  6. "Sun Microsystems makes Sun Grid Engine software available to Open Source community". Linox.com. 2001-07-23.
  7. Open Grid Scheduler
  8. Son of Grid Engine
  9. Templeton, Daniel (2010-12-23). "Changes for a Bright Future at Oracle". Retrieved 2011-01-19.
  10. "Univa Acquires Grid Engine Expertise" (Press release). Business Wire. 2011-01-18. Retrieved April 17, 2011.
  11. "Biography of Fritz Ferstl". Univa. Retrieved April 17, 2011.
  12. "Univa completes acquisition of Grid Engine assets" (Press release). enterpriseai. 2013-10-22. Retrieved October 22, 2013.
  13. "Univa Widens Grid Engine Gap" (Press release). businesswire. 2011-10-11. Retrieved October 11, 2011.
  14. "Univa Grid Engine 8.0.1 released". Daniel Gruber. 2011-10-03.
  15. "Univa Releases License Orchestrator Integrated with Univa Grid Engine to Reduce and Optimize Software License Expenses" (Press release). businesswire. 2013-06-18. Retrieved June 18, 2013.
  16. "Univa Grid Engine 8.1.5 and License Orchestrator 1.0.0". Daniel Gruber. 2013-07-25.
  17. "Univa Demonstrates Extreme Scale Automation by Deploying More Than One Million Cores in a Single Univa Grid Engine Cluster using AWS". Univa. 2018-06-24. Retrieved June 24, 2018.
  18. "Univa Grid Engine 8.0 Now Available" (Press release). Univa. 2011-04-12. Retrieved April 17, 2011.
  19. "Univa Grid Engine 8.0.0 Release Notes" (PDF). Univa. 2011-05-11.
  20. "Univa Grid Engine 8.0.1 Release Notes" (PDF). Univa. 2011-10-10. Retrieved December 10, 2012.
  21. "Univa Widens Grid Engine Gap". Business Wire. 2011-10-04.
  22. "Univa Grid Engine 8.1.7 Release Notes (covers 8.1.0 through 8.1.7)" (PDF). Univa. 2012-05-30. Retrieved January 15, 2014.
  23. "Univa Grid Engine 8.2.0 Release Notes" (PDF). Univa. 2014-08-26. Retrieved December 15, 2014.
  24. "Univa Grid Engine 8.3.1 Release Notes (covers 8.3.0)" (PDF). Univa. 2015-06-22. Retrieved November 11, 2015.
  25. "Univa Grid Engine 8.3.1 Release Notes (covers 8.3.0)" (PDF). Univa. 2015-06-22. Retrieved November 11, 2015.
  26. "New Version of Univa Grid Engine". www.univa.com. Retrieved 2016-06-13.
  27. "Univa Grid Engine 8.4.1 Release Notes (covers 8.4.0)" (PDF). Univa. 2016-06-05. Retrieved November 9, 2016.
  28. "Univa Grid Engine 8.5.0 Release Notes (covers 8.5.0 through 8.5.5)" (PDF). Univa. 2017-03-07. Retrieved January 24, 2018.
  29. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
  30. "Introducing Univa Grid Engine 8.6.1: Next-Level Enterprise Grade Workload Management". blogs.univa.com. Retrieved 2018-10-23.
  31. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
  32. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
  33. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
  34. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
  35. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
  36. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
  37. "Univa Grid Engine 8.6.12 Release Notes (covers 8.6.0 through 8.6.12)" (PDF). Univa. 2020-04-15. Retrieved April 15, 2020.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.