Technical report V1.0.2

Accuracy

Current V1.0.2 results: 2,048–4,096 taxa rerun in full; 32–1,024 taxa retained after matching current-version spot checks. Comparator accuracy is reused. Development data.

01

Gene-tree topology

Gene-tree mean nRF by method across all completed alignments; horizontal bars with 95% bootstrap confidence intervals.
PNG · SVG
02

Gene-tree branch lengths

Mean NMAE on each method’s correctly recovered internal branches, averaged across alignments. Lower is better.

Mean branch-length NMAE on each method’s correctly recovered internal branches; 95% whole-alignment bootstrap intervals.
03

Species-tree topology

Small · Simple

32–256 taxa
Small · Simple: methods ranked by mean nRF, with 95% whole-alignment bootstrap intervals.

Small · Complex

32–256 taxa
Small · Complex: methods ranked by mean nRF, with 95% whole-alignment bootstrap intervals.

Large · Simple

512–4,096 taxa
Large · Simple: methods ranked by mean nRF, with 95% whole-alignment bootstrap intervals.

Large · Complex

512–4,096 taxa
Large · Complex: methods ranked by mean nRF, with 95% whole-alignment bootstrap intervals.

Small uses shared completed alignments. Large includes all available scales; FastME * excludes 4,096-taxon timeouts. Data & coverage

04

Empirical data

DatasetConcordTree nRFFastTree nRFCASTER-site nRFDIPPER nRF
Oakleaf butterflies0.0571—0.02860.6970
1KP plants0.11230.09790.26130.9222
Papilionoid · plastid0.04560.02070.07050.2479
Fish · UCE0.06250.06250.15620.7097
Bembidion insects0.11630.11630.11630.5581
Mammals0.01680.00000.01260.0420
Ruminants0.04080.00000.04080.4845
Placental mammals0.15190.15190.13921.0000
Sigmodontinae0.12280.01750.14040.8641
Wolbachia0.03330.00000.16671.0000
Mean nRF0.07600.05190.11330.6526
05

Branch-support analysis

ROC curves for correct versus incorrect branches, ranked by local branch support.
TaskPooled AUC [95% CI]Coverage at display ≥95Correctness at display ≥95
Gene0.9343 [0.8824, 0.9763]92.35%99.63%
Species · 32–1,024 taxa0.9563 [0.9539, 0.9587]18.81%99.99%
06

Taxon completion

Paired topology ranking across all taxon sizes. Higher is better. Exact batch: the entire completed topology is correct.

Gene-tree completion

Gene-tree completion, 1 additions: methods ranked by paired relative topology rank score; exact batch recovery shown at right.
MethodRankExact
ConcordTree0.53290.0%
IQ-TREE constrained0.52789.8%
pplacer · sequential0.51486.6%
EPA-ng · sequential0.49182.8%
TIPars0.48982.2%
APPLES-2 · sequential0.44777.4%
Gene-tree completion, 4 additions: methods ranked by paired relative topology rank score; exact batch recovery shown at right.
MethodRankExact
IQ-TREE constrained0.58364.7%
ConcordTree0.57462.5%
pplacer · sequential0.53457.9%
EPA-ng · sequential0.48251.9%
TIPars0.45144.7%
APPLES-2 · sequential0.37541.9%
Gene-tree completion, 8 additions: methods ranked by paired relative topology rank score; exact batch recovery shown at right.
MethodRankExact
IQ-TREE constrained0.64746.2%
ConcordTree0.60141.4%
pplacer · sequential0.56238.4%
EPA-ng · sequential0.44826.3%
TIPars0.42721.9%
APPLES-2 · sequential0.31619.9%

Species-tree completion

Species-tree completion, 1 additions: methods ranked by paired relative topology rank score; exact batch recovery shown at right.
MethodRankExact
IQ-TREE fast + INSTRAL0.53494.6%
ConcordTree0.51691.7%
APPLES-2 · independent0.44984.5%
Species-tree completion, 4 additions: methods ranked by paired relative topology rank score; exact batch recovery shown at right.
MethodRankExact
ConcordTree0.59176.8%
IQ-TREE fast + INSTRAL0.55773.2%
APPLES-2 · independent0.35351.8%
Species-tree completion, 8 additions: methods ranked by paired relative topology rank score; exact batch recovery shown at right.
MethodRankExact
ConcordTree0.70160.1%
IQ-TREE fast + INSTRAL0.54938.7%
APPLES-2 · independent0.25019.0%
Metric & comparison protocol

On each input, a method earns 1 for each competitor it beats in whole-tree nRF, ½ for a tie, and 0 for a loss. We average across competitors, then equally across shared completed inputs. The score measures relative rank, not probability of correctness. Bars show 95% alignment-bootstrap intervals.

+1, +4 and +8 use separate paired panels. Gene methods use sequential placement, native TIPars or IQ-TREE constrained search. Species APPLES-2 and INSTRAL place independently; coincident queries remain unresolved, while ConcordTree can resolve relationships among new taxa. Exact batch recovery tests the complete output, including those relationships. INSTRAL uses IQ-TREE fast inferred gene trees; the true-gene-tree oracle is excluded. Results are descriptive development comparisons.