Don Lindberg, High performance computing and communications, and telemedicine

1.Historical background

In 1944, President Franklin D. Roosevelt wrote to Vannevar Bush DEng., head of the federal Office of Scientific Research and Development, and asked for recommendations for the post-World War II world. Dr. Bush’s 1945 response is the report “Science - The Endless Frontier [1]”. For “the war of science against disease”, Bush recommended that “the Government should extend financial support to basic medical research in the medical schools and universities”. The three other areas about which Roosevelt asked were scientific knowledge, aiding public and private research, and scientific talent. The 1950 creation of the National Science Foundation (NSF) traces its roots to Bush’s report.

Federal agency support for research and development (R&D) of advanced technologies has been a highly visible part of the American scene since the 1957 launch of Sputnik, the Russian satellite that put our nation on notice that it was no longer the unquestioned global leader in science and technology. In the public eye, the leading federal science agencies associated with the ensuing effort to re-establish our pre-eminence in these domains include NSF, the National Aeronautics and Space Administration (NASA) and the Department of Defense’s (DoD) Defense Advanced Research Projects Agency (DARPA) (both established in 1958), and the Department of Energy (DOE), which was established in 1977 by consolidating defense and civilian programs. Less visible Federal efforts include those by the National Security Agency, formed in 1952, which continued the codebreaking it had begun during World War II.

U.S. government support for advanced computing in the 1950s and 1960s slackened in the 1970s. When Donald A.B. Lindberg M.D. arrived at NLM in 1984, the Department of Health and Human Services (HHS) and its lead research organization, the National Institutes of Health (NIH) (roots date to 1887), were notably absent in the public perception of federal agency leadership and sponsorship of advanced computing. While appropriately and correctly viewed as an agency focused on health-related biological discovery and medical treatment research, the truth is that NIH also had been providing extramural support for R&D in biomedical computing and telecommunications since the 1960s. The NIH Computer Research and Biomathematics Study Section, a standing grant review committee, included Don Lindberg among its appointed members from 1967 to 1971. Notable achievements of the first several decades of NIH support included:

Each of these efforts could be considered “high performance computing” of their times, since this term has always been relative to whatever “ordinary performance computing” existed contemporaneously as a commonly available set of tools and resources. No clear boundary separates high performance computing and telecommunications from the spectrum of analogous commercially available devices, applications, and services, though the label is commonly reserved for systems that exceed commodity level performance by at least several orders of magnitude.

In 1984, the Division of Computer Research and Technology (DCRT) was the focal point for intramural computing at NIH. After NIH acquired its first digital computer as an experimental device in 1958, DCRT was established in 1963 and its first director, Arnold W. “Scotty” Pratt M.D., was named in 1966. Under Pratt’s leadership, DCRT built a campus computing infrastructure focused on time-shared mainframe and minicomputer hardware and software with a locally developed mainframe system called WYLBUR that supported administrative and scientific applications in NIH intramural labs, centers, and institutes [5].

With the completion of the 1985-1986 NLM Long Range Plan and setting of biotechnology information resources and services as a high priority, Dr. Lindberg oversaw the enhancement of NLM’s capabilities to serve new, computationally intensive activities such as gene sequence comparisons and alignment of macromolecules. In 1987, the Lister Hill National Center for Biomedical Communications (LHNCBC)—NLM’s existing computer R&D division where the new molecular biology tools and services were first created—established a T-1 (1.5 megabits per second) data network line to the National Cancer Institute’s (NCI) Advanced Scientific Computing Laboratory in Frederick, Maryland. That connection provided access to several scientific computers more powerful than NLM’s own machines, including a Cray X-MP 2 supercomputer, which at the time was approximately 400 times faster than LHNCBC’s VAX servers. It could execute 900 million floating point operations per second (FLOPS), or 90 percent of one gigaFLOP. Apropos of the evolving and relative nature of high-performance computing, as of this writing, current generation Graphics Processing Unit (GPU) cards for gaming PCs now routinely provide up to 20 trillion FLOPS (teraFLOPS) performance, which is approximately 20,000 times faster than the speed of the multi-million-dollar 1987 Cray supercomputer, at retail prices under $2,000.

Don Lindberg’s vision and conviction were that NLM should become a world leader in promoting and supporting advanced computing and telecommunications to realize its congressionally mandated role as an archive and distribution point for biomedical literature, data, and scientific knowledge in its many forms [6]. This global view of the opportunities and challenges presented by information technologies had been nurtured by his own career achievements and by serving on the National Academy of Sciences’ Computer Science and Engineering Board from 1971 to 1974 and on NLM’s Biomedical Library Review Committee from 1976 to 1980.

With his extensive prior experience on academic, professional society, and government advisory committees, Lindberg sought opportunities for NLM and NIH to engage in inter-agency computing and telecommunications collaborations. Part of the White House Office of Science and Technology Policy (OSTP), the Federal Coordinating Council for Science, Engineering and Technology (FCCSET), which had been established by Public Law (P.L.) 94-282 in 1976 to advise the President on science and technology matters and to coordinate federal science and technology efforts, provided one such opportunity [7]. The OSTP Director chaired the FCCSET, which included heads or their deputies from 13 federal departments and agencies involved in science and technology. Federal agency participation in FCCSET initiatives was voluntary. As a federal science agency, NIH had an open invitation to participate in cross-agency meetings and projects, about which the NIH Office of the Director informed Lindberg. The FCCSET was succeeded by the National Science and Technology Council.

In 1987, Lindberg assigned Daniel Masys M.D., director of NLM’s LHNCBC, to attend the regular monthly meetings of a nascent High Performance Computing and Communications (HPCC) Working Group of FCCSET’s Committee on Physical, Mathematical, and Engineering Sciences (CPMES). That Working Group provided both a conduit for understanding other agencies’ R&D efforts and a podium for presenting health-related “Grand Challenges”, i.e., important and difficult-to-achieve usage scenarios, as program goals. These biomedical research and healthcare “use cases” became a prominent feature of the interagency HPCC effort and were incorporated into the legislative record supporting passage of the High-Performance Computing Act in December 1991. Masys was subsequently named the NIH representative to the Working Group and served in that capacity until 1994 when he retired from federal service.

2.The HPC act and the HPCC program

FCCSET/CPMES published HPCC Grand Challenges reports to supplement the President’s Fiscal Year 1992 and 1993 Budgets [8,9]. They documented DARPA, DOE, NASA, NSF, the Department of Commerce’s National Institute of Standards and Technology (NIST) and its National Oceanic and Atmospheric Administration (NOAA), the Environmental Protection Agency (EPA), and NIH/NLM programs and plans. Lindberg kept his vow, made after the Global Change Research Act became law in 1990, that NLM would be explicitly included in any future multi-agency R&D program that had medical and human health impact. 70,000 copies of the 1993 report were printed, and it was said that “everyone” in Washington DC carried one.

Senator (later Vice President) Al Gore (D-TN) shepherded the High-Performance Computing (HPC) Act to passage, starting with the Supercomputer Network Study Act of 1986 [10]. That Act required OSTP to “report to the Congress on fiber optic networks… to improve communications among supercomputer centers and users”. The resultant “A Research and Development Strategy for High Performance Computing” reported agency views about supercomputer access, leadership, and research [11]. Between 1988 and 1991 seven HPC bills were introduced, there were seven amendments in the form of a substitute, and there were ten days of public hearings. NLM was mentioned in some versions of the bill, but the HPC Act calls for HHS and six other departments plus three independent agencies to report their HPCC programs and activities.

Dr. Masys testified at one such hearing, in September 1989 [12]. He described recent NLM computing and networking history. NLM had computerized the bibliographic Index Medicus database in 1964; had in 1971 created MEDLINE to make that database “available for on-line searching over public computer networks”; and by 1989 had 38 more databases online as part of MEDLARS (Medical Literature Analysis and Retrieval System), “the largest and most widely-used biomedical computing system in the world”. NLM researchers were then connected via DoD’s research-and-education-only network to NCI’s Cray. Masys stated that biology and medicine would need to understand and transmit images and videos, which was not feasible over the networks of those days; modeling protein folding for analyzing disease-causing agents was too slow on supercomputers of the day; and research in computerized tomography and magnetic resonance imaging required advances in both computing and networking. In answer to a question about how to accelerate progress, he said that vocabulary, language, and data interchange standards were needed, and cited NLM work on a Unified Medical Language System (UMLS).

In the end, the HPC Act of 1991 made HPC a Presidential-level initiative, with the Director of OSTP responsible for interagency coordination and annual reports to Congress [13]. The Act calls for an advisory committee to assess the program. It assigns agency responsibilities and authorizes appropriations from “sums otherwise authorized to be appropriated”. It calls for annual reports on any funds going to non-U.S. entities, and for a study on a supercomputer agreement with Japan.

OSTP established the HPCC Program and the National Coordination Office (NCO) for HPCC in 1992. The Program had four components—and major efforts:

NLM categorized its HPCC investments as follows:

3.Lindberg serves as HPCCIT subcommittee chair and director of the National Coordination Office for HPCC

In August 1992, OSTP Associate Director Eugene Wong, Ph.D., invited Dr. Lindberg to the Old (now Eisenhower) Executive Office Building (EOB). Lindberg wondered aloud to Kent Smith, his deputy, what “we” had done wrong. It was just the opposite. Upon his return from that meeting, Lindberg asked his executive assistant Pat Carson to cancel an extensive international trip so that he could devote his efforts to standing up the NCO. The cancellations took days.

It was said that a key reason Don Lindberg was asked to chair the FCCSET/CPMES HPCCIT (HPCC and Information Technology) Subcommittee and be NCO Director was that he represented an applications agency and not one of the “big four” technology agencies — DARPA, DOE, NASA, and NSF. Promoting grand challenge applications that could benefit from HPCC technologies, and not just the technologies themselves, was in fact key to passing the HPC Act. And the big four agencies considered themselves HPCC equals. Also, Lindberg was the only active HPCC agency principal with an international reputation. The HPCC community was flattered that someone of such prominence was asked to be the founding NCO Director.

In 1993, the Program added a fifth component, Information Infrastructure Technology and Applications, to develop the technology base for a National Information Infrastructure and to work with industry to develop and demonstrate prototype National Challenge applications to complement Grand Challenges [14].

Between 1992 and 1995, Lindberg brought health and medicine organizations into the HPCC Program: NIH’s National Center for Research Resources, DCRT, and NCI’s Biomedical Supercomputer Center; the Agency for Health Care Policy and Research; the Department of Veterans Affairs; and the Department of Education. Dr. Masys was the NIH HPCC coordinator. The HPCC Program’s FY 1996 annual report documented dozens of Grand Challenge and National Challenge applications [15]. The medical and health care grand challenges spanned biomedical imaging, molecular biology, and molecular dynamics, and included NLM’s Visible Human Project. National challenges included digital libraries, in which NLM reported MEDLARS and UMLS.

Lindberg took special pride in three specific HPCCIT meetings. Because the Presidential advisory committee authorized by the HPC Act had not yet been established, only one meeting with any particular private sector constituency was allowed. The first meeting was with directors of supercomputer centers, the second with independent software vendors, and the third with representatives from the telecommunications industry. Participants were asked to discuss their views, needs, and recommendations about the HPCC Program. These meetings typified the cooperative, collaborative relationships across HPCC agencies, industry, and universities.

The NCO was a lean operation. Three people were detailed from NLM: Pat Carson (then Jean Diehl after Carson went back to NLM), Charles Kalina as HPCCIT Executive Secretary, and a staff assistant. Cal Ramos was detailed from NASA and Sally Howe, Ph.D., from NIST. Space on B1 (B for basement) of LHNCBC’s Building 38A was outfitted for a large meeting room, a small meeting room, and offices. The HPCC Program had an early web site, and the NCO had a library and a librarian. Lindberg spent mornings in his NLM office on the mezzanine of NLM’s main Building 38 and afternoons at the NCO. A sign of the workload: he lost weight.

The HPCCIT Subcommittee met monthly. The NCO prepared detailed agendas and handouts with activity reports, invited speakers, and set up a long table for food -nutritious, as requested, and chocolate for the chair. The often more than 40 agency people who attended delighted in seeing each other and planning next steps. As agency representatives and alternates changed, Lindberg cordially welcomed newcomers, introducing them with spot-on comments about their accomplishments and interests.

Lindberg established a HPCCIT Executive Committee that included not just the big four agencies but also varying participation by smaller, generally applications agencies. That group helped plan HPCCIT Subcommittee meetings and identified larger issues before bringing them to the HPCCIT.

The April 1993 HPCCIT meeting materials included two items that Lindberg had written:

Dr. Lindberg announced at the January 1995 HPCCIT meeting that he would step down as NCO director in March. One could hear a pin drop; minutes seemed to pass as those present digested the news. In a January 25 letter, Vice President Al Gore thanked Lindberg for his “dedicated leadership [that] was instrumental in taking HPCC through its formative stages and in making it into the highly successful program which it has been,: and praised the “excellent work which government, industry and academia have produced under the auspices of HPCC to maintain our nation’s global leadership in science and engineering”. In March 1995, OSTP Director John H. Gibbons hosted a reception in the Old EOB’s ceremonial Indian Treaty Room honoring Lindberg’s service.

4.1995–2021: The next generation internet, health IT R&D, and HPCC today

The HPC Act of 1991 was first amended by the Next Generation Internet (NGI) Research Act of 1998 [16]. NGI goals (which they met) were experimental networking research, high-performance and ultra-high-performance connectivity testbeds, and revolutionary applications. DOE, NASA, NIST, and NSF had proposed a $100 million/year plus-up for themselves for FY 1998 and FY 1999. Lindberg wanted NLM included, so he added $5 million from NLM’s existing budget. Congress rewarded that act by adding $5 million to the NLM budget. That $5 million paid for the development of MedlinePlus.gov.

Lindberg testified at a 2000 NGI hearing [17]. He recalled that in 1996 Senator Bill Frist, a physician and chair of the subcommittee holding the hearing, did the first public MEDLINE search via the Internet. Between then and 2000, those searches had grown from 7 million per year to 250 million per year, with the general public fully a third of those users. He stated that both GenBank and the recently released ClinicalTrials.gov also needed the NGI.

The HPC Act of 1991 was also amended in 2007 and 2017 [18,19]. Changes include officially calling for periodic reviews and strategic plans and calling (again) for Grand Challenges. The amendments also changed the HPCC name; today it is the Networking and IT R&D (NITRD) Program. More than 20 agencies participate. It has expanded the number of components to include, for example, artificial intelligence, robotics, and wireless technologies.

Dr. Howe supported Dr. Lindberg from 1992 through 2015. She transferred from NIST to NLM in 1993, with the unwritten understanding that she could stay at the NCO. She moved with the NCO to NSF space in 1995 and remained until 2007 with titles Chief of Staff, Associate Director, and Acting Director. In the early 2000s she supported Lindberg in his position as a co-chair of the NITRD Health IT R&D (HITRD) Interagency Working Group (IWG), the Program’s first and only applications-oriented IWG, in which fifteen agencies now participate. Upon her 2007 return to NLM, she was assigned to an office whose shelves happened to hold old HPCC materials, which she organized. With Lindberg’s permission, she successfully asked that the NCO return the NCO library, said yes to more than a dozen cartons labeled NCO/HPCC found in NLM basements (including the medicine and computing articles dated 1936–1994 that Lindberg had moved from his NLM office to his NCO office) and built an archive. While Lindberg’s guidance was to “save everything”, he seemed chagrined at 48 cartons that measured 59 linear feet, more than the usual for some notable physicians. The archive is officially the NCO/HPCC archive because that is an official Government name. In 2017, Howe completed the archive’s public-facing finding aid [20].

Today’s NITRD Program (nitrd.gov) still has the same overall structure and the same inclusive and trusting agency relationships that Lindberg had nurtured. Beyond their contributions to medicine and health (including COVID-19 R&D), thirty years of steady HPCC/NITRD R&D have created, enabled, and/or improved climate modeling and weather forecasting; energy sources and energy efficiency; environmental understanding; the Internet and search engines; national defense and intelligence; remote and STEM education and training; space exploration; wireless networking and cell phones; and more. They are all Lindberg’s HPCC legacy.

5.HPCC and telemedicine at NLM

From 1987 when NLM first participated in HPCC activities through his 2015 retirement, Dr. Lindberg was a liaison between the healthcare and HPCC communities. Telemedicine was a key focus area that combined health care and communications, the second C in HPCC.

Lindberg initiated an NLM research contract program to provide examples of how physicians could practice better medicine by using advanced computing and networking capabilities along the “Information Superhighway”. Twelve multi-year HPCC Health Care Awards were made in 1994 in four areas: testbed networks to link hospitals, clinics, medical schools, and libraries to allow for sharing of medical data and images; collaborative technology for real-time treatment of patients; information access; and virtual reality for medicine [21]. Many of these awards demonstrated the use of advanced high-speed networks.

NLM sponsored a 1995 study, “Telemedicine: A Guide to Assessing Telecommunications for Health Care”, by the Institute of Medicine (IOM) (now part of the National Academies of Science, Engineering, and Medicine) [22]. It documented “the benefits and costs of this blend of medicine and digital technologies” so that “cautious decision-makers” might knowledgably invest in its use and development.

Shortly after Lindberg stepped down as NCO Director and returned to being the full time NLM Director, he recognized that the healthcare and health research communities still needed to play a role in the HPCC Program. His solution was to establish the Office for HPCC (OHPCC) within the Office of the NLM Director. For agencies or activities involved in healthcare or health-related research, whether they were part of the HPCC Program or not, OHPCC became a resource for HPCC information and guidance.

Dr. Lindberg asked Michael Ackerman, Ph.D., to serve as the Assistant Director of NLM for HPCC and as Chief of the new OHPCC office. While OHPCC was organizationally part of LHNCBC, it had a direct connection to Lindberg’s office. On one of Lindberg’s early visits to OHPCC, Ackerman inquired about Lindberg’s vision for HPCC and OHPCC; he needed to draft a position description that reflected Lindberg’s expectations. Ackerman sat pen in hand ready to take notes. Lindberg said that notes were not needed: “The job description is very simple. Do good deeds and other duties as assigned”. With that, he turned and left.

This anecdote illustrates Dr. Lindberg’s relationship with most of the people he hired. He had the ability to find knowledgeable, smart, and creative people whom he could trust. He supported them and gave them great freedom. However, he wanted to be kept informed and was always eager to get involved when needed.

As OHPCC Chief, Ackerman represented NIH in the HPCC Program’s Large Scale Networking (LSN) Working Group. This group comprised agency representatives conducting or funding technical research to support the future Internet. While NLM itself didn’t conduct or fund such research, Lindberg knew that NLM could provide practical, real-world healthcare testbeds for evaluating advanced networking technologies. Nonetheless, early on, NLM’s presence in LSN and HPCC was barely tolerated. That lasted until the Program gave one of its annual in-person presentations for the U.S. Congress. NLM had the last position on the agenda, and during his presentation Ackerman explained how the technologies described by his HPCC colleagues would be used in healthcare. The reaction of the Congressional panel was “Why didn’t they just say so?” After that, NLM was always placed towards the beginning of the agenda.

With the IOM Telemedicine Study in hand, NLM next funded demonstrations of advanced telemedicine through its National Telemedicine Initiative. Anticipating projects that might be proposed, Lindberg knew that sending personal health information over the nascent Internet would require security and privacy protection. NLM sponsored a National Research Council (NRC) study to examine “technological and organizational aspects of security management, including basic principles of security; the effectiveness of technologies for user authentication, access control, and encryption; obstacles and incentives in the adoption of new technologies; and mechanisms for training, monitoring, and enforcement”. “For the Record: Protecting Electronic Health Information” was published in 1997 [23]. That report included recommendations that ultimately formed the basis for many Health Insurance Portability and Accountability Act (HIPAA) regulations.

In 2004, National Telemedicine Initiative Awards were made for 19 projects. Demonstrating his ability to promote and spotlight NLM projects, Dr. Lindberg invited then-HHS Secretary Donna Shalala to announce the awards. “Telemedicine offers us some of our best and most cost-effective opportunities for improving quality and access to health care. The projects we are supporting will evaluate the use of telemedicine in a wide variety of settings, all the way from the care of newborns and children with disabilities, to the elderly and chronically ill, and those needing a range of specialist care”, Secretary Shalala said [24]. “These are imaginative and well-targeted projects that will help us determine how we can best use information via telemedicine for clinical decision-making [24]”.

All journals indexed in NLM’s MEDLINE are vetted by NLM’s Literature Selection Technical Review Committee (LSTRC). At that time, eligible journals had to have their articles independently peer reviewed. Lindberg knew that most telemedicine articles appeared in “grey literature” and therefore couldn’t be included in MEDLINE. As part of the National Telemedicine Initiative, NLM supported the Telemedicine Information Exchange (TIE) both to support the telemedicine community and to experiment with “grey literature” citations [25]. Many of the publications cited by TIE were later included in MEDLINE.

The additional funding provided when the NGI Research Act became law in 1998 gave Lindberg the means to encourage the healthcare community to experiment with futuristic possibilities for patient care. Lindberg and Ackerman gave many presentations in anticipation of NLM’s next funding opportunity. “If we are to benefit from the fruits of modern medical science we must be able to transfer massive amounts of data—instantaneously, accurately, and securely”, said Lindberg [26]. NLM’s goal was to demonstrate how the NGI could “affect health care, health education, and health research systems in such areas as cost, quality, usability, efficacy, and security”. It was clear that Lindberg’s vision was way beyond the imagination of the health community, so NLM’s NGI initiative was divided into two phases. Phase I invited short proposals to explore the possibility and feasibility of using NGI technologies to provide healthcare in ways that were then not possible, and helped raise healthcare community awareness. Phase II invited proposals for implementing and testing the kinds of ideas generated by Phase I. Twenty-four Phase 1 awards were made in 1998 and fifteen Phase 2 awards were made in FY 2000 [27,28]. The funded projects were designed around NGI capabilities expected to become available in the not-too-distant future: virtually error-free service; security and medical data privacy; “nomadic” computing; network management; and infrastructure technology for “collaboratories”. Twenty years later, these qualities are part of everyday networks.

The Internet was evolving rapidly then, and so was NLM’s view of healthcare’s Internet future. The NLM Scalable Information Infrastructure (SII) program was established in FY 2003 as a continuation of Phases I and II of the NLM NGI program. SII supported the research priorities in the President’s Information Technology Advisory Committee’s (PITAC) 1998 report “Information Technology Research: Investing in Our Future [29]”. (PITAC was the Presidential advisory committee called for in the HPC Act.) The SII program’s purpose was “to encourage the development of health-related applications of scalable, network aware, wireless, geographic information systems, and identification technologies in a networked environment”. SII focused “on situations that require, or will greatly benefit from the application of these technologies in health care, medical decision-making, public health, large-scale health emergencies, health education, and biomedical, clinical, and health services research”. Projects had to use networks “linking one or more of the following: hospitals, clinics, health practitioners’ offices, patients’ homes, health professional schools, medical libraries, universities, medical research centers, laboratories, or public health authorities”. Eleven awards were made [30].

Over the 15 years since NLM’s HPCC telemedicine programs ended, many of their demonstration projects and the methods derived from them were further developed. Since the spring of 2020 the world has been dealing with the COVID-19 pandemic, and in doing so has depended heavily on the Internet, now thought of as broadband networks, and on telemedicine. Thanks to those decades of R&D, telemedicine technologies were readily available, and they were widely used for both routine healthcare and care unique to the pandemic. Dr. Lindberg was correct, “Do good deeds”.