The world is about to need more energy. A lot more. The combination of providing basic energy services to emerging markets and powering a new generation of data centers and manufacturing activity means the era of flat energy demand is over. Grid operators all across the United States are grappling with a rapid uptick in load growth projections and scrambling to build the energy infrastructure necessary to meet those forecasts.
As the world enters this new phase of energy growth, one thing is certain: the United States must lead the world and pioneer the technologies to make it possible. America is blessed with a world-class energy innovation engine led by the Department of Energy and the seventeen National Labs that have launched countless new technologies into the market. The Department of Energy (DOE, the Department) is the world’s largest funder of research for physical sciences and applied energy research, development, and demonstrations (RD&D). It is the sole federal entity with the capacity to advance innovative clean energy technologies in coordination with the private sector. These public-private partnerships are critical to commercializing breakthrough technologies domestically and ultimately exporting them to key partners around the globe.
Congress has recently enacted legislation to reinvigorate these efforts, providing billions of dollars in new funding to support innovative energy demonstration projects and the commercialization of new technologies. The conditions are right for America to meet this challenge, but the next presidential administration must agree to prioritize energy deployment first and foremost.
Hydraulic Fracturing:
A Public-Private R&D Success Story
The transition to natural gas generation has been driven by the abundance of inexpensive gas produced from unconventional resources—such as shale formations—using a combination of directional drilling and hydraulic fracturing. The gas production rate from hydraulically fractured wells increased from less than five billion cubic feet per day in 2000 to over fifty billion cubic feet per day in 2015—over two-thirds of total U.S. gas production.1 This production from unconventionals provides approximately $100 billion of benefit to consumers each year.
Natural gas was first extracted from unconventional formations as early as the 1820s, but high costs made widespread deployment uneconomic. Advancements were needed in three principal technological areas in order to improve the economics of unconventional gas extraction, including directional drilling and drill bits, hydraulic fracturing, and subsurface imaging to facilitate resource characterization.
Integration of these advancements was then required to complete the process. Government support of R&D starting in the 1970s drove down costs of unconventional extraction and led to the economic benefits we see associated with natural gas today.
The efforts of oil-industry entrepreneur George Mitchell and his company Mitchell Energy are often celebrated as launching the successful commercialization of unconventional natural gas production. While Mitchell deserves great credit, both for unusual persistence and for the ultimate development of an approach that combined horizontal drilling with hydraulic fracturing, there were other factors that enabled the commercial success of gas extraction from unconventionals.2 One of the most significant was government-funded research and development.
When federal programs were first initiated in the 1970s, private sector R&D funds were declining, and industry was unwilling to take on the risks of ongoing, unproductive exploration using experimental methods. The federal government did not take over private sector innovation, but instead lowered risks associated with the energy technology development process described above.
By the mid-1970s, the United States had become increasingly import-dependent for oil. The Arab Oil Embargo caused an energy crisis and exacerbated economic stagnation in a time of inflation. These challenges led to the creation of the Energy Research and Development Administration (ERDA), which later became the Department of Energy.3 One of ERDA’s goals was to promote access to unconventional gas resources in order to overcome declining gas production.4 In 1975, ERDA partnered with industry to drill the first Appalachian Basin directional wells to tap shale gas and the first horizontal shale well to tap seven individual hydraulically fractured intervals.5 In addition, two federal engineers patented an early directional shale drilling technique that allowed operators to span larger radial expanses of shale deposits. Building from these early successes, ERDA launched additional programs focused on the next wave of key issues.
Federally supported basic research was critical to technology advancement, and government participation in technology demonstrations lowered industry exploration risks and showed the market the emissions reduction potential of unconventional gas. Alex Crawley, a former program director at ERDA who oversaw a significant portion of the federal research that went into the development of shale gas fracturing technologies, described the outlook of the industry at the time, saying, “as far as shale is concerned, I don’t know that industry would have ever taken a look at it without the federal program.”6 While the private sector—e.g., Mitchell Energy—generally drove applied R&D, there were multiple instances throughout the development process when federal applied R&D made significant contributions and led to additional innovation. Dan Steward from Mitchell Energy acknowledged the broad role played by DOE in helping Mitchell commercialize technology and the vital role of government at various stages in the development process. “They helped us to evaluate how much gas was there and evaluate the critical properties as compared to the Devonian shale of the Appalachian basin. They helped us with our first horizontal well.7 They helped us with pressure build-ups. And we worked with them on crack mapping.”
Similarly, in the 1970s, DOE supported R&D led by Sandia laboratory focused on the development of polycrystalline diamond compact (PDC) drill bits for high-density, high-temperature applications needed to drill geothermal wells. These drill bits could cut harder rock, drill faster, and last longer. Sandia partnered with General Electric to develop laboratory test procedures and computer models of drill bit performance. Results from these laboratory tests and Sandia’s computer modeling, as well as further industry collaborations, spurred additional applied research and development and eventual commercialization by private industry. A recent study estimates that PDC drill bits yielded cost savings of $15.6 billion from 1982 to 2008, with half of this added value attributed to the DOE’s investment of $26.5 million.8
In the early 1990s, DOE worked with several companies to conduct publicly subsidized, cost-shared demonstration projects in which several wells containing multiple hydraulically fractured zones were completed, including Mitchell Energy’s famous first horizontal well in the Barnett Shale in 1991. These efforts were the first demonstrations of the multiple‑fracture drilling method in gas shales that would become the norm for all shale gas recovery.9
Public-private partnerships played a critical role in these efforts. They ensured that R&D was effectively directed and helped speed the diffusion of research findings and associated technologies.10 The partnership extended to funding as well. Estimates of funding to support technology development and commercialization efforts are summarized in figure 2. As noted, early funding for basic R&D was dominated by direct DOE expenditures. As technologies matured, public-private partnership became the dominant source of funding. Data regarding direct industry funding are not available, but gas production provides a good surrogate for spending by industry. As noted, gas production increases rapidly as the technology approaches market readiness, at which point DOE and Gas Research Institute (GRI) funding ends and industry moves the technology into full commercial operation.
Global events and markets hinted that something needed to be done regarding natural gas supplies in the early 1970s. This prompted the initiation of R&D in the late 1970s that led to successful commercialization in the 2000s. Making it through such a long process required vision and persistence on the part of both government policymakers and scientists as well as commercial interests such as Mitchell Energy. Because the process took so long, there were many opportunities to abandon the effort along the way. In the end, vision and persistence proved worthwhile. As noted above, shale gas provides approximately $100 billion of benefit to consumers each year. Even if the cost estimates included in figure 2 are low by a factor of two, the expenditures were clearly an excellent investment in terms of both time and money. These fracturing technologies have been instrumental in propelling the United States to become the world’s top producer of oil and natural gas.
Recent reports from the Energy Information Administration (EIA) have indicated a 20 percent decline in U.S. energy-related carbon emissions from 2005 to 2023. The main driver behind this emissions reduction is that more electricity has been generated from natural gas than from other fossil fuels—natural gas is a less carbon-intensive fuel than either coal or petroleum.11
The improvements made in drilling technologies, hydraulic fracturing, and microseismic imaging that were supported by government programs and initiatives clearly played a pivotal role in advancing technologies that were ultimately integrated and commercialized by industry and that have led to the abundance of U.S. natural gas.
The story of hydraulic fracturing shows that early-stage government investment in R&D can reduce the risks that industry is unwilling to take on. These commitments allow public-private partnerships to thrive in terms of both investment and research direction throughout the development process. This process is often a long-term endeavor and can ultimately result in significant economic benefits.
The Department of Energy’s Evolving Mission
Today, the United States faces none of the same conditions that characterized the energy crisis of the 1970s, yet the legacy structure of the Department largely persists. As early as 1996, the General Accounting Office (GAO, subsequently renamed the Government Accountability Office), found that the Department’s culture, organizational structures, and management practices remained deeply rooted in a model based on the Cold War, with distributed sites and laboratories around the country.12
In recent years, the United States has transformed from an import-dependent country to become a net energy exporter since 2019. Most notably, this era of American energy dominance has been marked by the United States becoming the world’s largest producer of oil and gas. The core challenges facing the U.S. energy sector today are how to best promote American technology at home and abroad, advance energy innovation, and thwart the influence of foreign adversaries over energy and mineral supply chains.
The Department of Energy needs to remain focused on U.S. global leadership in energy production, advancing innovative energy technologies, protecting national security interests, and supporting fundamental research and science. While the Department was originally charged with national energy policy and energy research in a time of crisis, congressional interest in consistently funding those functions waned as oil prices came down in the 1980s.
Over time, the Department has been assigned a variety of missions, including an increasing focus on the Cold War and nuclear weapons development in the 1980s, remedying environmental cleanup failures from decades of legacy nuclear weapons waste in the early 1990s, addressing national security failures in the late 1990s at the national laboratories that called into question the Department’s continued existence, and, in the early 2000s, early-stage, fundamental research and science came to be viewed as the Department’s core strength.
The Department has now entered a fifth phase. Congress has, in recent years, expanded the Department’s energy innovation mission, providing unprecedented funding increases to commercialize new technologies through demonstration programs. These new authorities stem from bipartisan legislation, including the Energy Act of 2020, the chips and Science Act, and the Infrastructure Investment and Jobs Act (IIJA). If implemented effectively, these programs could reduce emissions, lower energy costs to consumers, boost domestic manufacturing, and allow the United States to retain its position as a global energy leader.
These bills represent core parts of the Republican energy innovation agenda. Five years in the making, the Energy Act of 2020 was the first major energy reauthorization since the Energy Policy Act of 2005. It was the bipartisan culmination of dozens of individual energy bills that were passed and consolidated into a package granting the Department new authorities, reauthorizing programs such as arpa-e, and providing new direction on decarbonization and demonstration programs. These bills were built on the successful launches of the Advanced Reactor Demonstration Program (ARDP) and the Energy Storage Grand Challenge catalyzed under the Trump administration. Effective implementation will require the Department to prioritize energy security, innovations for major emitting sectors, and public-private partnerships to achieve rapid deployment.
The challenge ahead demands a rethinking of how to best align the applied energy offices to implement the energy research, demonstration, and deployment mission enacted by Congress. The current structure of the Department encourages political appointees and career officials alike to advocate for specific technologies rather than promoting an integrated, practical application of technology innovation in the energy sector. In recent years, administrations of both parties have tweaked the Department’s structure while leaving the organization of the applied energy offices largely unchanged.
The next secretary of energy should be empowered with the necessary tools to lead strategically from day one. A key focus must be to foster coordination within the Department, emphasizing the importance of adapting to ensure long-term durability. My organization, ClearPath, recently published recommendations for the next presidential administration in its “Proposal to Modernize the U.S. Department of Energy.”13
Opportunities for the Next Presidential Administration
(1) Develop procedures to protect private American intellectual property, ensuring Department funding for energy innovation by U.S. entities in collaboration with our allies while protecting national security from China and other adversaries.
In order for the United States to lead the next generation of clean energy technologies and control their supply chains, the Department must protect American IP and national security rather than enabling Chinese state actors to control U.S. supply chains for advanced technologies. This requires the Department to ensure that technologies in development do not leak through state-sponsored efforts, industrial espionage, or neglect. A recent GAO report found that DOE lacks guidance or binding requirements related to foreign acquisition risks.14 Equally important, the Department must identify pathways to reshore U.S. intellectual property that has been compromised due to past failures.
Developing integrated policies and processes requires effective management from the top. The next secretary must make it clear that national security is a top priority for applied energy programs and national laboratories alike. Processes and controls must be consistently followed. Administered correctly, IP controls provide freedom for scientific and research endeavors while protecting American property rights. A comprehensive approach will entail a top-to-bottom review of existing processes, including enforcement, and the previous use of any waiver authorities especially in regard to China.
Aligning, strengthening, and streamlining IP structures across the DOE complex will provide more certainty for the private sector actors seeking to work with the Department. Collaborating with national laboratories should be reliably seen as a value-added activity for American companies and research institutions. Clarity in this regard will ensure that American innovation is invigorated while international scientific collaboration in the U.S. national interest is not inadvertently chilled.
(2) Streamline Department funding competitions to encourage new applicants. The Department should minimize application requirements that are unrelated to the technical merits of the project to ensure funding decisions are free from political or social influence.
The Department should seek ways to remove unneeded complexity to make it easier to contract and partner with the Department. By making applications more straightforward and removing bureaucratic requirements, the Department will increase the diversity of applications, types of entities applying, and innovative approaches being introduced. At the same time, the Department must tighten its review processes to ensure that awards do not inadvertently create a national security threat or compromise energy security. Funding competitions have been straightforward in the past for organizations and researchers seeking to apply. In general, applicants must demonstrate a sound technical approach, innovative solution, and the requisite personnel to carry it out. For demonstration projects, financial feasibility must also be factored in before selections are made.
While support from the community remains critical to long-term success for future deployments of new technologies, community benefit plans should not be a significant portion of the quantitative evaluation criteria. The new emphasis on mandatory community benefit plans, with the Department assigning as much as 20 percent of the evaluation for demonstration projects, is deleterious for innovative new technologies. In some cases, the weighting is equal to the technical evaluation sections. The Department should instead remain focused on assessing technological innovation, due diligence, and fiscal propriety—not promoting social policy.
Placing significant emphasis on mandatory community benefit plans is unnecessary to determine which projects are best positioned for federal demonstration project funding. The Department should not direct industry on how to do projects or mandate certain human resources policies.
(3) Implement “early-success, early advancement” or “rapid prototyping” initiatives to accelerate promising, high-impact technologies. The Department should identify R&D with proven early success and then provide additional funding to achieve well-defined milestones in a rapid timeframe. Setting aside a percentage of discretionary funding for rapid innovation will promote end-to-end innovation, complementing other funding tools such as arpa-e.
Rapid prototyping can accelerate promising developments based on well-defined sprint objectives, moving high-impact technologies, materials, and novel methods rapidly toward commercialization. Rapid prototyping can also address the gap between initiatives with similar goals but tightly restricted participants, such as the scaleup program at arpa-e, which is restricted to past arpa-e or other DOE awardees and Laboratory-Directed Research and Development (LDRD) projects at the national laboratories.15 This approach has the potential to be more flexible than the Department’s existing mechanisms and would respond better to scientific, engineering, and technology developments as they occur.
Illustrative examples for rapid prototyping exploration could include fusion energy technologies, novel materials for extreme environments, and aspects of quantum computing. Rapid prototyping complements, rather than replaces, other mechanisms like Small Business Innovation Research (SBIR) funds, LDRDs, or arpa-e activities. In recent years, the Department has increased the use of prizes and American Made Challenge competitions and the scaleup program through arpa-e to leverage alumni technologies for additional follow-on funding. There is significant opportunity across program offices to foster end-to-end innovation, including rapidly identifying and advancing promising innovations as they arise. Importantly, rapid prototyping should not cannibalize funding for scaleup, SBIR, or LDRDs.
(4) The Department must coordinate with other federal agencies and use every tool at its disposal to expedite permitting for demonstration projects, which frequently require authorizations from multiple agencies. Accordingly, the Department should designate a director-level position within the Office of the Secretary to manage interagency permitting needs to prevent bureaucratic delays. Further, the Department should extend its existing R&D categorical exclusion to include demonstration projects.
The Department will need to manage permitting requirements closely with awardees, local governments, and communities for demonstration projects. Many of the large demonstration projects will require environmental permitting approvals from multiple agencies. While the Department may issue funding awards, the Department does not control the entirety of the permitting process for its awardees. Therefore, success will depend on interagency coordination to ensure projects do not suffer from avoidable delays.
The Federal Permitting Improvement Steering Council (FPISC) was created in 2019 to address this challenge, but it has not been as effective as hoped. Concurrently, the Council of Environmental Quality (CEQ) and the National Economic Council (NEC) in recent years have not played as significant of an interagency coordinating role as they did during the Bush and Trump administrations. This coordination should be a significant focus of the energy and environment portfolio within the Executive Office of the President.
Federal permitting difficulties have now come to the forefront as a chokepoint for new energy projects and technologies, particularly in regard to the National Environmental Policy Act (NEPA).16 Demonstration projects and those backed by federal guarantees should stand or fall based on technological and economic feasibility, not delays in federal permitting and interagency bureaucracy.
One area where the Department can accelerate permitting is through its existing categorical exclusion for Research and Development activities, which in its current form specifically excludes demonstration actions. The Department should extend this categorical exclusion to include demonstration projects funded by the Department.
DOE will be severely tested because of the volume and complexity of demonstration projects. Further, many of them include novel technologies and configurations that have not been reviewed before. The Advanced Reactor Demonstration Program (ARDP) offers an example of the challenging permitting path ahead for other major demonstration programs. Launched and awarded under the Trump administration prior to the Energy Act of 2020 and IIJA, The ARDP has experienced a lack of coordination between DOE and other federal agencies, namely the Nuclear Regulatory Commission (NRC). Without clear guidance, these first-of-a-kind projects face additional bureaucratic risk. Due to outstanding questions and concerns raised by the ARDP projects, the agencies ultimately signed a memorandum of understanding (MOU) for interagency coordination. Because of these ongoing risk factors, the Department needs to centralize management and coordination at the highest level to mitigate future problems for the additional demonstration programs. A failure to fully address these risks would inevitably lead to a loss of congressional support for future demonstration project funding. The Department’s priority should be to avoid creating unnecessary hurdles for these projects. Appointing a director in the Office of the Secretary to manage timelines and performance of the portfolio at large, along with individual projects, will be a critical factor for success. This is in stark contrast to current practices, under which NEPA is administered across offices throughout the DOE complex.
Addressing these elements ensures that the proposed reorganization aligns with the current energy landscape while learning from historical perspectives. In total, the recommendations in this report will advance the U.S. position as a global energy leader and promote fiscally responsible policy. Together, they will best position the Department to deliver on innovation priorities enacted by Congress.
This article originally appeared in American Affairs Volume VIII, Number 3 (Fall 2024): 104–14.
Notes
1 Robert M. Amers et al., “
The Arithmetic of Shale Gas,” Yale Graduates Energy Study Group, June 15, 2012.
2 John M. Golden and Hannah. J. Wiseman, “The Fracking Revolution: Shale Gas as a Case Study in Innovation Policy,” Emory Law Journal 64, no. 4 (2015): 955–1040.
3 National Energy Technology Laboratory, “DOE’s Unconventional Gas Research Programs 1976–1995,” U.S. Department of Energy, January 31, 2007.
4 Jason Burwen and Jane Flegal, “Unconventional Gas Exploration & Production,” American Energy Innovation Council, March 2013.
5 Golden and Wiseman, “The Fracking Revolution.”
6 Alex Tremblath, “Interview with Alex Crawley, Former Program Director for the Energy Research and Development Authority,” Breakthrough Institute, May 21, 2012.
7 Michael Shellenberger, “Interview with Dan Steward, Former Mitchell Energy Vice President,” Breakthrough Institute, December 12, 2011.
8 Golden and Wiseman, “The Fracking Revolution.”
9 National Energy Technology Laboratory, “DOE’s Unconventional Gas Research Programs.”
10 Burwen and Flegal, “Unconventional Gas Exploration & Production.”
11 Michael Shellenberger et al., “Where the Shale Gas Revolution Came From: Government’s Role in the Development of Hydraulic Fracturing in Shale,” Breakthrough Institute, May 23, 2012.
12 U.S. Congress, Senate, Committee on Energy and Natural Resources, Department of Energy: Observations on the Future of the Department, testimony of Victor S. Rezendes, 104th Cong., 2nd sess., 1996.
13 Matthew Mailloux and David Solan, “ClearPath Proposal to Modernize the U.S. Department of Energy,” ClearPath, March 2024.
14 Government Accountability Office, “Department of Energy: Actions Needed to Assess U.S. Manufacturing Policy and Protect Technology from Foreign Acquisition,” Government Publishing Office, May 23, 2024.
15 U.S. Department of Energy Advanced Research Projects Agency–Energy, “The Scaleup Program,” accessed July 10, 2024.
16 Matthew Mailloux, “Clean Energy Permitting Could Go from Bad to Worse,” ClearPath, September 6, 2023.
