{"id":1156,"date":"2022-06-27T15:43:34","date_gmt":"2022-06-27T15:43:34","guid":{"rendered":"https:\/\/www.hammerdb.com\/blog\/?p=1156"},"modified":"2022-07-05T16:49:48","modified_gmt":"2022-07-05T16:49:48","slug":"hammerdb-for-managers","status":"publish","type":"post","link":"https:\/\/www.hammerdb.com\/blog\/uncategorized\/hammerdb-for-managers\/","title":{"rendered":"HammerDB for Managers"},"content":{"rendered":"<p>This post is targeted towards the questions most often asked by non-technical management who want to get up to speed on what HammerDB is (what it isn&#8217;t) and how it can benefit their organization.<\/p>\n<h3>What is HammerDB?<\/h3>\n<p>HammerDB is a software application for database benchmarking.\u00a0 It enables the user to measure database performance and make comparative judgements about database hardware and software.<\/p>\n<p>HammerDB has graphical and command line interfaces for the Windows and Linux operating systems.<\/p>\n<p><a href=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/45screen.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-1176 alignleft\" src=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/45screen-300x248.png\" alt=\"\" width=\"300\" height=\"248\" srcset=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/45screen-300x248.png 300w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/45screen-1024x847.png 1024w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/45screen-768x635.png 768w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/45screen.png 1224w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbcli.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1182 alignleft\" src=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbcli-297x300.png\" alt=\"\" width=\"300\" height=\"303\" srcset=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbcli-297x300.png 297w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbcli-100x100.png 100w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbcli.png 499w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<h3>Why HammerDB was developed<\/h3>\n<p>Databases are highly sophisticated software, and to design and run a fair benchmark workload is a complex undertaking. The Transaction Processing Performance Council (TPC) was founded to bring standards to database benchmarking, and the history of the TPC can be found <a href=\"https:\/\/www.tpc.org\/information\/about\/history5.asp\">here<\/a>. The TPC designed benchmarks for transaction processing (OLTP) and analytics (OLAP) and anyone can run these benchmarks, have them audited by the TPC and published on the official benchmark rankings.<\/p>\n<p>However, although these results are the gold standard of database benchmarking to do so, requires time, expertise and not insignificant cost. Additionally, many databases contain a license clause colloquially known as a <a href=\"https:\/\/en.wikipedia.org\/wiki\/David_DeWitt\">De-Witt<\/a> clause that prevents the publication of non-approved benchmarks.<\/p>\n<p>These factors meant that often when looking for database performance information, the results for a particular combination of software and hardware were not available.<\/p>\n<p>As the TPC makes its specifications available for free, the need was seen for an open source benchmarking application that could leverage the standards written by the TPC yet implemented quickly, easily and at low cost by anyone. Also, when testing a database with a De-Witt clause, it was then possible to produce your own results without having to rely on a particular vendor to publish results of interest.<\/p>\n<h3>The HammerDB name<\/h3>\n<p>Originally, HammerDB was named\u00a0 Hammerora because the first database the application supported was Oracle. Ora was a common prefix\/suffix for Oracle related software, and the name inspired by a <a href=\"http:\/\/www.hammerfilms.com\/\">genre of classic films<\/a> and the characters portrayed, in particular for testing Oracle RAC.\u00a0 As more databases were added, the original name was less appropriate and the name HammerDB was suggested and adopted.<\/p>\n<h3>HammerDB development<\/h3>\n<p>HammerDB was originally developed by <a href=\"https:\/\/www.linkedin.com\/in\/steve-shaw-80b6aa20\/\">Steve Shaw<\/a> as an employer approved\u00a0 own-time, own-materials project to implement workloads derived from TPC specifications in a user accessible way.\u00a0 An important concept was to simulate database users called Virtual Users in parallel (rather than concurrently) to accurately simulate a real database workload with multiple users running from separate systems. Programming languages such as Java, Lua and Python were unable to support this unique requirement (an application to simulate separate users in multiple threads that do not block or pause each other&#8217;s workloads, the Python GIL for example only permits one user to run at any one time) and the design decisions made are discussed in the following post.<\/p>\n<blockquote class=\"wp-embedded-content\" data-secret=\"83eBA7RQ06\"><p><a href=\"https:\/\/www.hammerdb.com\/blog\/uncategorized\/what-programming-languages-does-hammerdb-use-and-why-does-it-matter\/\">What programming languages does HammerDB use and why does it matter?<\/a><\/p><\/blockquote>\n<p><iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" title=\"&#8220;What programming languages does HammerDB use and why does it matter?&#8221; &#8212; HammerDB Blog\" src=\"https:\/\/www.hammerdb.com\/blog\/uncategorized\/what-programming-languages-does-hammerdb-use-and-why-does-it-matter\/embed\/#?secret=5uoFuEw74p#?secret=83eBA7RQ06\" data-secret=\"83eBA7RQ06\" width=\"525\" height=\"296\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe><\/p>\n<p>As a result, HammerDB was able to scale beyond other database workload applications as the following post illustrates.<\/p>\n<blockquote class=\"wp-embedded-content\" data-secret=\"ZlaPFr5mY7\"><p><a href=\"https:\/\/www.hammerdb.com\/blog\/uncategorized\/why-tcl-is-700-faster-than-python-for-database-benchmarking\/\">Why Tcl is 700% faster than Python for database benchmarking<\/a><\/p><\/blockquote>\n<p><iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" title=\"&#8220;Why Tcl is 700% faster than Python for database benchmarking&#8221; &#8212; HammerDB Blog\" src=\"https:\/\/www.hammerdb.com\/blog\/uncategorized\/why-tcl-is-700-faster-than-python-for-database-benchmarking\/embed\/#?secret=dVUwlMIO2s#?secret=ZlaPFr5mY7\" data-secret=\"ZlaPFr5mY7\" width=\"525\" height=\"296\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe><\/p>\n<h3>Adoption by the TPC<\/h3>\n<p>Given the increasing importance of open source and the widespread use of HammerDB, the TPC adopted HammerDB in 2019 and now hosts the project on <a href=\"https:\/\/github.com\/TPC-Council\/HammerDB\">GitHub<\/a>. Today, the TPC-OSS subcommittee oversees development and approves all modifications made through pull requests.<\/p>\n<h3>Usage and industry adoption<\/h3>\n<p>HammerDB maintains a web page under the <a href=\"https:\/\/www.hammerdb.com\/stats.html\">stats link<\/a> where the number of downloads can be tracked. HammerDB is used globally, with the areas of use shown in blue on the map below with a darker color showing higher levels of use, with the most popular destinations being the USA and China.<\/p>\n<p><a href=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbww.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-1184\" src=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbww-300x171.png\" alt=\"\" width=\"300\" height=\"171\" srcset=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbww-300x171.png 300w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbww-768x438.png 768w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/hdbww.png 808w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>The results of HammerDB workloads have been published by all leading cloud vendors, database software vendors and systems suppliers, HammerDB maintains a <a href=\"https:\/\/www.hammerdb.com\/benchmarks.html\">collated list<\/a> of these publications but does not vet or audit results before inclusion.<\/p>\n<h3>HammerDB Licensing<\/h3>\n<p>HammerDB is open source software licensed under the<br \/>\n<a href=\"https:\/\/www.gnu.org\/licenses\/gpl-3.0.en.html\">The GNU General Public License v3.0<\/a> (GPLv3) and a quick guide to GPLv3 can be viewed <a href=\"https:\/\/www.gnu.org\/licenses\/quick-guide-gplv3.en.html\">here<\/a>.\u00a0 HammerDB has dependencies on external open source software and can be built from source.<\/p>\n<blockquote class=\"wp-embedded-content\" data-secret=\"jFMlnu2ZsN\"><p><a href=\"https:\/\/www.hammerdb.com\/blog\/uncategorized\/how-to-build-hammerdb-from-source\/\">How to build HammerDB from source<\/a><\/p><\/blockquote>\n<p><iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" title=\"&#8220;How to build HammerDB from source&#8221; &#8212; HammerDB Blog\" src=\"https:\/\/www.hammerdb.com\/blog\/uncategorized\/how-to-build-hammerdb-from-source\/embed\/#?secret=JySznR3bh4#?secret=jFMlnu2ZsN\" data-secret=\"jFMlnu2ZsN\" width=\"525\" height=\"296\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe><\/p>\n<p>HammerDB is Free software and consequently engineers should consider not only how they can benefit from using the software but also how they can contribute to the community with code and documentation.<\/p>\n<p>Businesses that depend on open source should consider sponsorship of open source projects or financial support to ensure that the open source they depend on remains freely available.<\/p>\n<h3>Supported Databases<\/h3>\n<p>HammerDB supports the most popular databases on the <a href=\"https:\/\/db-engines.com\/en\/ranking\">db-engines ranking<\/a>, namely Oracle Database, Microsoft SQL Server, IBM Db2, TimesTen, MySQL, MariaDB, PostgreSQL, Greenplum, Postgres Plus Advanced Server, Citus Data, Amazon Aurora and Amazon Redshift.\u00a0 HammerDB supports these databases running in the cloud and in the enterprise, and will also run workloads against databases derived from the most popular open source databases MySQL and PostgreSQL.<\/p>\n<p>The wide range of database support gives HammerDB an advantage over other database benchmarking tools that only implement workloads against one or two databases, limiting comparison between database engines and assessing relative performance.<\/p>\n<h3>Derived Workloads<\/h3>\n<p><strong>\u00a0<\/strong>When testing database performance there are 2 distinct workloads, transactional or OLTP and analytic (data warehouse, decision support) or OLAP.\u00a0 HammerDB supports 2 workloads derived from TPC specifications to test these different requirements, namely TPROC-C derived from TPC-C for OLTP and TPROC-H derived from TPC-H for OLAP.<\/p>\n<p>It is important to note that TPC-C and TPC-H are registered trademarks of the TPC and using the names TPC-C or TPC-H and\/or as official metrics such as tpmC or QphH in a non-audited publication is considered a trademark violation and should not be used.<\/p>\n<p><a href=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/oltpolap.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-1188\" src=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/oltpolap-184x300.png\" alt=\"\" width=\"184\" height=\"300\" srcset=\"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/oltpolap-184x300.png 184w, https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2022\/06\/oltpolap.png 269w\" sizes=\"auto, (max-width: 184px) 100vw, 184px\" \/><\/a><\/p>\n<p>An additional specification called TPC-CH for hybrid transactional\/analytical processing (HTAP) is under research and development for inclusion in a future release\u00a0 as TPROC-CH.<\/p>\n<p>In addition to the TPC-C specification for OLTP workloads, the TPC has also developed the TPC-E specification.\u00a0 HammerDB will consider developing a TPC-E derived workload when official benchmark publications are made from at least 3 of the supported databases to ensure a fair representation in an open source version.<\/p>\n<h3>The NOPM Metric<\/h3>\n<p>When reporting TPROC-C workloads the key metric is known as NOPM or New Orders per Minute.\u00a0 This metric measures the same value as the tpmC metric in an official TPC-C publication, however as noted previously the use of official terminology in derived workloads is not permitted and therefore HammerDB uses NOPM as a derived metric.\u00a0 HammerDB also reports TPM as an engineering metric, it is not a requirement for there to be a fixed relationship between NOPM and TPM across databases and therefore NOPM can be used for comparison of performance\u00a0 between databases whereas TPM is for analysing a particular database engine.<\/p>\n<h3>Cached vs Scaled Workloads<\/h3>\n<p>The official TPC-C OLTP workload is a what is known as a <em>scaled<\/em> workload, however a key difference in the HammerDB design was by default to implement a smaller more efficient <em>cached<\/em> workload based on the same specification that could still give an indication of comparative performance in the same way that the<em> scaled<\/em> workload could.<\/p>\n<p>A key difference between cached and scaled workloads is the implementation of <em>keying<\/em> and <em>thinking<\/em> time to introduce a pause of time between transactions. In a scaled derived TPC-C workload allowing for this keying and thinking time, one Virtual User will complete approximately 1 New Order per Minute and therefore for example 10,000 database sessions will run at approximately 10,000 NOPM and 100,000 sessions at 100,000 NOPM.\u00a0 The workload also outputted the data from the Virtual Users by simulating individual terminals.<\/p>\n<p>Note that HammerDB can also implement a scaled workload with a feature called event-driven scaling.\u00a0 However, this requires a large data set and middleware to manage the large database session count.\u00a0 Instead, most users prefer to implement a cached workload.<\/p>\n<p>When HammerDB was designed, it was clear that where the database software was scalable (initially such as Oracle), CPU performance at full utilization was the key determining factor for database performance. Therefore, a perfectly scaled implementation provided the levels of memory and I\/O or disk capacity to reach full CPU utilization.<\/p>\n<p>However, prior to the advent of high performance Solid State Disks (SSDs) to implement such a configuration required a high capacity of hard disk drives (HDDs) in one or many fibre attached storage arrays at considerable expense.<\/p>\n<p>Instead, HammerDB implemented a cached workload by eliminating the keying and thinking time and the requirement for terminals. Now individual Virtual Users don&#8217;t pause between transactions and can run at tens of thousands NOPM each.\u00a0 With the reduced I\/O footprint, the data each Virtual User requires is much reduced, meaning that most of the workload is cached in memory. This means we can reach full CPU utilization (with scalable database software) quicker and without requiring middleware.<\/p>\n<p>This approach gives us an indication of what an optimally configured scaled configuration with a high I\/O and memory capacity can achieve, however be aware that for a production environment you will have measured the CPU&#8217;s database potential, but you will need sufficient memory, I\/O and scalable database software to make full use of this CPU potential.<\/p>\n<h3>Summary<\/h3>\n<p>1. HammerDB is a software application for database benchmarking.<\/p>\n<p>2. HammerDB was developed to allow anyone to run database benchmarks quickly, easily and at low cost.<\/p>\n<p>3. HammerDB was designed to scale and is developed in a language that is not restrictied by a Global Interpreter Lock (GIL) that restricts workloads to being single-threaded and instead runs in parallel.<\/p>\n<p>4. The HammerDB name was inspired by a <a href=\"http:\/\/www.hammerfilms.com\/\">genre of classic films.<\/a><\/p>\n<p>5. HammerDB is hosted by Transaction Processing Performance Council (TPC) who oversee development.<\/p>\n<p>6. HammerDB is used globally and provides statistics on downloads and publications.<\/p>\n<p>7. HammerDB is open source software licensed under the<a href=\"https:\/\/www.gnu.org\/licenses\/gpl-3.0.en.html\"> The GNU General Public License v3.0<\/a> (GPLv3).<\/p>\n<p>8. HammerDB supports the most popular relational databases on the <a href=\"https:\/\/db-engines.com\/en\/ranking\">db-engines ranking.<\/a><\/p>\n<p>9. HammerDB runs workloads called TPROC-C and TPROC-H derived from the TPC specifications TPC-C and TPC-H respectively with the NOPM metric, the key metric for measuring transactional performance. Using TPC terminology for non-audited benchmarks violate TPC trademarks.<\/p>\n<p>10. By default HammerDB implements a <em>cached<\/em> vs a <em>scaled<\/em> workload but can implement both types of benchmark.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This post is targeted towards the questions most often asked by non-technical management who want to get up to speed on what HammerDB is (what it isn&#8217;t) and how it can benefit their organization. What is HammerDB? HammerDB is a software application for database benchmarking.\u00a0 It enables the user to measure database performance and make &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.hammerdb.com\/blog\/uncategorized\/hammerdb-for-managers\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;HammerDB for Managers&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"ppma_author":[5],"class_list":["post-1156","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"authors":[{"term_id":5,"user_id":2,"is_guest":0,"slug":"hammerdb","display_name":"HammerDB","avatar_url":{"url":"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2018\/10\/logo-white.png","url2x":"https:\/\/www.hammerdb.com\/blog\/wp-content\/uploads\/2018\/10\/logo-white.png"},"author_category":"","user_url":"http:\/\/www.hammerdb.com","last_name":"","first_name":"","job_title":"","description":""}],"_links":{"self":[{"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/posts\/1156","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/comments?post=1156"}],"version-history":[{"count":43,"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/posts\/1156\/revisions"}],"predecessor-version":[{"id":1251,"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/posts\/1156\/revisions\/1251"}],"wp:attachment":[{"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/media?parent=1156"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/categories?post=1156"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/tags?post=1156"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.hammerdb.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1156"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}