To ensure the effective support of its broad user community, NSF-supported centers have conducted annual user surveys for many years. In August and September 2005, the annual survey was conducted for the first time as a joint activity of the Cyberinfrastructure Partnership (SDSC and NCSA) and the TeraGrid. This year's survey not only showed the continued success of the NSF-supported cyberinfrastructure environment, but also demonstrated the user community's general satisfaction with the resources, software and services from all NSF cyberinfrastructure sites.
The survey responses are summarized in Table 1 below. Satisfaction scores are given on a scale of 1–5, with 5 being "extremely satisfied." The table shows the top five responses for questions; unless otherwise indicated, the table shows the number of respondents and the percentage of respondents selecting that answer.
The survey was open for four weeks during August and September 2005. The 39 questions covered the areas of computational environment, grid environment and services, programming environment, software and tools, data and I/O, visualization, allocations, consulting and help desk support, training and documentation, and some optional demographic information.
We advertised the survey by sending e-mail messages to active users at NCSA, SDSC, PSC and TeraGrid. A total of 412 complete and partial responses were recorded. CIP and TeraGrid sites will make decisions about resources, services, and software based on this survey.
In the survey, users also had the opportunity to report on the scientific successes they have achieved thanks to NSF-supported resources, and about half took that opportunity. While these results are more qualitative, they do underscore the broad impact that NSF Cyberinfrastructure has across many scientific disciplines. The following user quotes are representative of the successes reported:
- "We found (to our surprise) that linear alkynes do not have the same stabilization due to conjugation that alkenes have. This has stirred up some activity (and some controversy) in the computational study of hydrocarbon bond interactions, bond energies, and in the general nature of hydrocarbon bonds. A classic example of a new surprise in a field in which 'everything is already known.'"
- "We have managed to perform a large number of high-dynamical-range simulations of the formation of the first generation of stars in the universe in both a standard cosmology and using warm dark matter particles. In addition, we have followed the evolution of the HII regions (ionized gas) from these stars, and also their supernovae. This work is groundbreaking and will be published in upcoming issues of The Astrophysical Journal.
- "Scattering and propagation properties of electrically large objects have been analyzed using fast integral equation methods in time and frequency domain. Range of new functionalities of the methods have been introduced. Results published in IEEE Transactions on Antennas and Propagation and reported at IEEE APS Symposiums."
- "The computational resources allowed us the simulation, for the first time, of heat transport in thin silicon films. This research is of relevance in nanoelectronics. It was published in refereed journals and presented at numerous conferences."
- "We have completed a number of successful research projects. É This research relates to effects of human activities on stratospheric ozone, future responses of ozone to international policy, effects of climate change on air quality in the U.S., potential impacts of climate change on the Midwest and more recently on the N.E. United States."
- "It enables me to study the hot gas inside the bulge-dominated galaxies for the first time. The results of the simulation are used to compare the state-of-the-art observation from CHANDRA and XMM-Newton, providing detailed physical understanding of the nature of the hot diffuse gas inside the galaxies."
- "[We] identified Arabidopsis promoter elements implicated in diurnal control of gene expression."
- "Many publications on strongly correlated electron systems such as the high-temperature superconducting cuprates."
- "Most of my computations are being performed using the NSF cyberinfrastructure. We have used Molecular Dynamics calculations to understand protein-DNA interactions and obtained very interesting results, which will be published soon. I also have used these facilities to understand various structural and energetic properties of DNA and RNA, the results of which have been published/submitted for publication."
- "Impact of Head-Disk Slider on the head disk. Was able to characterize the thermo-mechanical behavior of the head-disk interface under shock loading."
- "Simulations of solar magneto-convection on granular and supergranular scales have helped clarify the nature of faculae, the excitation of oscillations, the validation of local helioseismic inversion techniques, the driving of convection, the emergence of magnetic fields in the quiet Sun."
- "We investigate the generation and evolution of the intermediate scale size irregularities in high-latitude plasma patches, which are of interest to the High Latitude Plasma Structure group of the NSF-sponsored Coupling, Energetics, and Dynamics of the Atmospheric Regions initiative. We have convincingly demonstrated the nature of the mesoscale structuring of high-latitude plasma patches by a combination of instabilities starting with the classical gradient drift instability (GDI) and secondary Kelvin-Helmhotz instability followed by a tertiary generation of a self-consistent shear flow."
- "We can simulate turbulent mixing at the highest Reynolds number up to date which helps resolve longstanding problems in turbulence research. Also we can follow Lagrangian particles in a turbulent flow at high Reynolds number."
The survey indicates that the NSF Cyberinfrastructure programs are highly successful in meeting the overall needs of a very diverse user community. Highlights from the survey include:
- 40% of the respondents associated themselves with chemistry, molecular and cellular biology, and physics, but at least one respondent identified with each NSF division and directorate.
- Users are generally satisfied with the offered compute resources, mass storage systems and site environments, with most major compute platforms, storage systems and sites scoring 4 or better.
- The user community uses a diverse collection of software and tools for their research. Regarding third-party software used (an open response question), the 145 respondents identified more than 100 different applications and tools across the NSF-supported cyberinfrastructure.
- Users are expanding their use of grid capabilities, although 'Don't know' tops the list of grid tools users are using or would be interested in using, 32% of respondents are using or interested in using MPICH/MPICH-G2 and 22% are using or interested in GridFTP. More generally, nearly 27% of users are starting to experiment with Grid tools, and another 28% may use them within two years; 18% have conducted actual research with grid tools and services.
- Users are general satisfied with the allocations process (average rating of 4.1) and the allocations and consulting/help desk services (average ratings of 4.2 or better). E-mail (72%) and phone (15%) remain the most popular means of contacting help desk staff.
The following results indicate a need to increase awareness of the opportunities and resources available through CIP and TeraGrid. The CIP and TeraGrid teams will use this input to help prioritize efforts in the coming year:
- For Data and I/O, 'scp' is the most popular data management tool (cited by 52% of respondents), with Matlab (25%) and archival storage system commands (24%) in second and third. These responses indicate that users may benefit from greater awareness of the data movement and management options available.
- While there was general satisfaction with training opportunities by those users who had participated, the majority of respondents had not taken advantage of the training offered. Other questions indicated the need for training in certain areas. Documentation, while most users were satisfied (average rating of 3.9), could use some improvement in order to match the satisfaction levels seen in other areas.
- 'None' is the top response (34%) among math software and libraries being used in user codes. FFTW and LAPACK tied for second (23%). More awareness of the performance benefits of using optimized libraries is needed.
- Similarly, 'None' is by far the top response (47%) for the parallel/performance tools used. This indicates a need for awareness about and training in the use of the available debugging and performance analysis tools.
- 'None' is the also top visualization software used (47%), with Matlab (19%) and Gnuplot (18%) distant second and third. And the visualization resources offered by NSF Cyberinfrastructure sites are notable mostly for having 86% or greater 'N/A' responses. Users need to be educated as to the potential of visualization software and use of available visualization resources.
Table 1. Summary of User Survey Results
| COMPUTE RESOURCES |
Responses |
Avg. Rating |
| ANL TeraGrid IA64 Cluster |
32 |
3.7 |
| Caltech TeraGrid IA64 Cluster |
35 |
3.5 |
| Indiana AVIDD-I64 Cluster |
17 |
3.4 |
| NCSA Condor Pool (radium) |
15 |
3.7 |
| NCSA IBM p690 (copper) |
106 |
4.0 |
| NCSA SGI Altix (cobalt) |
64 |
3.9 |
| NCSA TeraGrid IA64 Cluster |
95 |
4.2 |
| NCSA Xeon Cluster (tungsten) |
100 |
4.1 |
| ORNL Xeon Cluster |
11 |
3.4 |
| PSC HP Marvel (Rachel) |
50 |
3.9 |
| PSC TCS1 (Lemieux) |
98 |
4.0 |
| Purdue IBM SP |
14 |
3.5 |
| Purdue Linux Clusters |
14 |
3.5 |
| SDSC DataStar p655 |
67 |
4.2 |
| SDSC DataStar p690 |
64 |
4.1 |
| SDSC TeraGrid IA64 Cluster |
70 |
4.0 |
| TACC Cray-Dell Clstr (Lonestar) |
20 |
3.8 |
| TACC Sun Fire E25K (Maverick) |
10 |
3.4 |
| SITE, OVERALL |
Responses |
Avg. Rating |
| TeraGrid (overall) |
148 |
4.0 |
| NCSA |
210 |
4.3 |
| PSC |
116 |
4.1 |
| SDSC |
131 |
4.2 |
| ANL/UC |
33 |
3.8 |
| Caltech |
29 |
3.4 |
| Indiana U |
19 |
3.4 |
| ORNL |
18 |
4.1 |
| Purdue U |
15 |
3.5 |
| TACC |
18 |
4.2 |
| MASS STORAGE SYSTEMS |
Responses |
Avg. Rating |
| NCSA UniTree |
132 |
4.1 |
| PSC FAR |
70 |
4.0 |
| SDSC HPSS |
80 |
4.1 |
| TACC DMF |
10 |
3.6 |
| SRB |
32 |
3.8 |
| GridFTP-UniTree |
33 |
3.7 |
| GRID ENVIRONMENT AND SERVICES |
| Grid tools using/interested in: |
Experience with/plan to use grid tools: |
Don't know (101; 33%)
MPICH/MPICH-G2 (97; 32%)
GridFTP (67; 22%)
None (52; 17%)
Globus command line (43; 14%) |
May use w/in 2 years (88; 28%)
Starting to experiment (84; 26%)
No plan to use (80; 26%)
Experienced (57; 18%) |
Advanced Portal Features desired
(No. Responses, Interest 1-3 scale): |
Metaqueue (301; 2.6)
Reserve resources (294; 2.4)
Manage staging of data (296; 2.2)
Discussions w/users (297; 2.1) |
| PROGRAMMING SOFTWARE AND TOOLS |
| Parallelization Method: |
Math Software/Libraries |
MPI (176; 61%)
None (46; 16%)
OpenMP/MPI (39; 13%)
OpenMP (30; 10%)
Don't know (24; 8%) |
None (98; 34%)
FFTW (67; 23%)
LAPACK (66; 23%)
Matlab (53; 18%)
IBM ESSL (40; 14%) |
| Parallel/Perform. Tools |
Third-party Software |
None (127; 47%)
MPI_wtime() (47; 17%)
TotalView (47; 17%)
Don't know (44; 16%)
gdbx (27; 10%) |
Gaussian (30)
CHARMM (12)
NAMD (12)
AMBER (10)
IDL (10)
|
| DATA AND I/O |
| Software, Tools Used: |
Useful Persistent Disk |
scp (157; 52%)
Matlab (74; 25%)
Arch stor cmds (72; 24%)
None (46; 15%)
Mathematica (40; 13%) |
1 TB or less (153; 52%)
1 TB – 10 TB (103; 35%)
10 TB – 50 TB (28; 10%)
50 TB or more (12; 4%) |
| Persistent Disk Access by |
Use Data Collections |
Network file sys. (193; 69%)
GridFTP (53; 19%)
Other (21; 7%)
SRB (14; 5%) |
No (237; 82%)
Yes (51; 18%) |
| VISUALIZATION |
| Software Used: |
None (132; 47%)
Matlab (52; 19%)
Gnuplot (50; 18%)
IDL (34; 12%)
VMD (30; 11%) |
| ALLOCATIONS |
Responses |
Avg. Rating |
| Overall Process |
266 |
4.1 |
| Allocations Staff |
247 |
4.3 |
| Online help/documentation |
254 |
3.9 |
| Proposal format/length |
219 |
3.9 |
| POPS |
187 |
4.1 |
| Time to get award |
210 |
3.9 |
| Reviewer Comments |
171 |
3.7 |
| CONSULTING AND HELP DESK SUPPORT |
| Phone Consulting |
Responses |
Avg. Rating |
| TeraGrid Help Desk |
83 |
4.2 |
| NCSA Consulting |
96 |
4.4 |
| NCSA Help Desk |
99 |
4.4 |
| SDSC Consulting |
78 |
4.3 |
| E-mail Consulting |
Responses |
Avg. Rating |
| help@teragrid.org |
95 |
4.2 |
| consult@ncsa.uiuc.edu |
116 |
4.4 |
| help@ncsa.uiuc.edu |
118 |
4.4 |
| consult@sdsc.edu |
91 |
4.3 |
| Preferred Contact Method |
E-mail (217; 72%)
Telephone (44; 15%)
Interactive text chat (15; 5%)
Discussions/'bugzilla' (9; 3%)
Web forms (8; 3%) |
| TRAINING AND DOCUMENTATION |
| TRAINING |
Responses |
Avg. Rating |
| On-site Courses |
66 |
4.1 |
| Access Grid Courses |
33 |
3.9 |
| Video Archive |
20 |
3.5 |
| Web-based Courses |
39 |
4.1 |
| DOCUMENTATION |
Responses |
Avg. Rating |
| Web sites |
261 |
3.9 |
| User Guides |
236 |
3.9 |
| User News |
189 |
3.7 |