Results from 100 arXiv (first hundred from June 2015). A hundred is nowhere near enough for a reliable analysis of this sort, because the timing of articles placed in arXiv is far from random. Perhaps best to add a +-5%.
mollwollfumble said:
For months I’ve been concerned that I have difficulty telling the difference between good science and bad science in the latest technical articles in astrophysics. Perhaps I’ve go so behind the times that I can no longer understand technical science papers. Or perhaps not, it occurred to me just now that perhaps I’m running into Sturgeon’s Law.
Sturgeon’s law is “90% of everything is crap”. But is it really 90%?
Results.
A) Crap science (13%)
5% 1) Papers on ramifications of scientific hypotheses that have already been disproved (including those rejected by Occam’s razor).
1% 2) Observations that are not statistically significant.
3% 3) Hobby horse science – the same paper published over and over again with only trivial modifications.
2% 4) Minimal effort, would have required less than three man-days to research.
2% 5) Just plain wrong.
B) Papers that, while not complete crap, do not advance science much (55%)
3% 1) Just another paper about @#$%. ie. a hobby horse but on a global scale.
1% 2) Minutiae that add only a trivial amount of new information.
35% 3) Confirming what is already known (or confirming that nobody knows) to no more accuracy than earlier studies.
0% 4) Timid science, so afraid of offending people that it doesn’t come to any conclusions.
4% 5) An un-approved proposal.
12% 6) A mathematical method that isn’t very new.
C) There’s average science (30%)
13% 1) New observations or mathematical modelling of something that nobody cares about.
12% 2) New observations that supersede older ones.
0% 3) A new science facility.
1% 4) Confirming a discrepancy between theory and observation.
4% 5) New (unconfirmed) hypothesis or new observations that could reduce the discrepancy between theory and observation.
D) There’s great science (2%)
2% 1) Observed for the first time, and important.
0% 2) New statistically significant discrepancy between theory and observation.
0% 3) A newly-confirmed theory about something important.
E) Other:
0% 1) I haven’t got a clue what they’re talking about.
In summary, to apply Sturgeon’s law to scientific articles in hard science, you need to replace that “90%” by “about 70%”.
Some of the more “interesting” articles.
1% A2) Observations that are not statistically significant.
Observations of TiO2 in the atmosphere of a giant star
3% B1) Just another paper about @#$%. ie. a hobby horse but on a global scale.
I expected heaps of “black hole” papers here, but they mostly avoided me this time. The first of the two I did count as global hobby horses in this dataset is the current obsession with variability of active galactic nuclei. It’s been known for at least 40 years that this is intrinsically unpredictable, as all the latest papers confirm. The second is theories of cosmic inflation, anything not known about cosmic inflation from the results of the Planck space telescope will remain unknown for a very long time. Not counted as a global hobby horse but surprisingly popular is the magnetic field in intergalactic space, I gather this was recently discovered to be much stronger than expected, but now everyone agrees why.
B3) Confirming that nobody knows.
The most startling example of this is a paper confirming that we haven’t a clue how much oxygen is in the Sun. Spectral line evidence is grossly inconsistent, and even more inconsistent unless we have the Sun’s nickel content wrong, and helio-seismicity doesn’t clarify the issue.
0% B4) Timid science.
I had that on the list because that’s a description of the papers describing the final cosmological results from the Planck satellite.
12% C2) New observations that supersede older ones.
I have two favourites here. In one, combining data from SDSS and WISE found that our previous estimation of the star formation rate in the universe was too high by a factor of 1.4. The other is a 15 month result from Pan-STARRS, which includes absolute magnitudes of 250,000 asteroids.
1% C4) Confirming a discrepancy between theory and observation.
The atmosphere of Pluto is denser, contains more nitrogen, than it should. Keep a watch on the results from New Horizons to see if Pluto has any nitrogen volcanos.
2% D1) Observed for the first time, and important.
Supernovae type 1a are the “standard candles” of the cosmos. For the first time we have a time-series of spectrographs from a nearby example, in the Cigar galaxy M82.
0% E1) I haven’t got a clue what they’re talking about.
(Pat self on back).