The paper presents results from the Collaborative Computational Project in Wave Structure Interaction (CCPWSI) Blind Test Series 2. Without prior access to the physical data, participants, with numerical methods ranging from lowfidelity linear models to fully nonlinear Navier–Stokes (NS) solvers, simulate the interaction between focused wave events and two separate, tautmoored, floating structures: a hemisphericalbottomed cylinder and a cylinder with a moonpool. The ‘blind’ numerical predictions for heave, surge, pitch and mooring load, are compared against physical measurements. Dynamic time warping is used to quantify the predictive capability of participating methods. In general, NS solvers and hybrid methods give more accurate predictions; however, heave amplitude is predicted reasonably well by all methods; and a WECSim implementation, with CFDinformed viscous terms, demonstrates comparable predictive capability to even the stronger NS solvers. Large variations in the solutions are observed (even among similar methods), highlighting a need for standardisation in the numerical modelling of WSI problems.
Numerical modelling is a crucial part of the design analysis in most engineering disciplines, particularly offshore engineering. Recently, with the continued industrialisation of the seas, engineers are readily turning to floating solutions to maximise exploitation. Consequently, there exists an overwhelming number of numerical codes to accommodate this transition and provide quantitative descriptions of the interaction of waves with floating systems. When selecting a numerical tool for a task, the model fidelity is a theoretical metric, indicating the degree of simplification adopted in the physics, as well as the corresponding mathematical model. The highfidelity Navier–Stokes (NS) model is theoretically more accurate than lower fidelity models – for example, the fully nonlinear potential theory (FNPT), due to its lower degree of simplification in the physics (capable of including multiphase processes as well as the effects of viscosity, turbulence and compressibility). The practical fidelity and reliability of these models, however, depends on the specific implementation – for example, the discretisation of the governing equations, which in turn relies heavily on the experience of the operator. Further, more sophisticated methods tend to come with considerable penalties in terms of the required computational resource (prohibiting their use in routine engineering design), so typically a compromise has to be made. Therefore, although considerable overlaps in predictive capability are likely, in general, selecting an appropriate (let alone the optimal) numerical model (or implementation) is not trivial, particularly because the physical requirements of the model might not be known from earlier. To complicate things further, what constitutes an acceptable result is also both case and operatorspecific – that is, the operator might prioritise speed over accuracy or vice versa. Further still, as discussed by Ransley et al. (2019, 2020) and as a consequence of this complexity, there is no established, or appropriate, method to quantify predictive capability. Consequently, in terms of mitigation against failures due to neglected physical phenomenon (or, conversely, against the use of excessive computing resource), developing new analysis methods to understand and quantify the predictive capability of these models is of great value to the engineering industry. This is the motivation for the Collaborative Computational Project in Wave Structure Interaction (CCPWSI) Blind Test Workshops.
The CCPWSI Blind Test Workshops bring together numerical modellers from the WSI community with the goal of assessing current numerical methods and accelerating the development of numerical modelling standards. In these workshops, volunteers are invited to simulate a set of test cases, based on bespoke laboratory experiments covering a range of physical complexities, without prior access to the measured data. It is believed that, the ‘blind’ nature of the workshops provides a better assessment of a method's predictive capability than traditional comparative studies, in which participants can ‘tune’ their results to achieve the best possible match with the physical data. Further, to support contributions using all types of WSI codes, no constraints are applied to the computational implementation, and participants are encouraged to follow ‘best practice’ procedures. As was found in the preceding CCPWSI Blind Test Series’ (Ransley et al., 2019, 2020), however, there can be considerable differences in the implementation of a numerical model (based on operator preferences), and similar codes can display very different capabilities depending on the strategy used. This complicates the comparison, compared to studies in which a specific implementation is enforced, and these blind tests could be criticised for promoting ‘competition’, rather than collaboration, between participants. Despite this, these blind tests highlight a key risk to industry end users and reinforce the need for bestpractice guidelines in WSI modelling as well as provide evidence of what constitutes superior implementation (that would otherwise be lost if this were constrained). They also offer additional insight that complements similar, alternative efforts (Wendt et al., 2019).
The CCPWSI Blind Test Series 2 is, chronologically, the third series within the CCPWSI blind test workshops and is held in conjunction with the European Wave and Tidal Energy Conference (EWTEC). Series 2 (along with Series 3 (Ransley et al., 2020)) considers the motion response and mooring load of floating bodies and is built on the basis of the CCPWSI Blind Test Series 1 (Ransley et al., 2019), in which the capability of participating models is compared in terms of the pressure and runup on a fixed FPSO model. The release of the Series 2 test cases was made in September 2018, and the showcase event was held during a series of special sessions and a side event at the 13th EWTEC in Napoli, Italy (1–6 September 2019). For more information on the CCPWSI blind test workshops (CCPWSI, 2020a).

Geometry 1  Geometry 2  

Mass: kg  43.674  61.459 
0.191  0.152  
1.620  3.560  
1.143  3.298  
Draft: m  0.322  0.330 
Mooring rest tension: N  32.07  31.55 
−0.131  −0.178 
Positive

Probe  1  2  3  4  6  7 

−1.50  −2.00  −2.75  −4.25  1.50  2.00  
0  0  0  0  0  0 
Probe  8  9  10  11  12  13 

2.75  −1.50  0  1.50  0  0  
0  −1.50  −1.50  −1.50  −2.00  −2.75 
Probe 5 corresponds to the position of the structure's CoM in cases with the structure included.

Case  

1BT2  0.25  0.3578  3.0  0.274  0.128778 
2BT2  0.25  0.4  3.0  0.274  0.160972 
3BT2  0.25  0.4382  3.0  0.274  0.193167 
The CCPWSI Blind Test Series 2 involved 30 volunteers from 13 institutions. There were 11 submissions ranging from linear potential theory (LPT) to NS solvers, including; hybrid (coupled) methods, partialparticle methods, finiteelement methods (FEM), finite difference methods (FDM) and finite volume methods (FVM); opensource, commercial and inhouse codes. Each method is described and summarised in Table 4.

Code reference  Discretised scheme  Theory  Freesurface treatment  Turbulence treatment 

PIC (inhouse)  FDM + meshless  NS  MAC + (1phase)  Laminar 
OpenFOAM (source term)  FVM  NS  VOF  Laminar 
Hybrid FNPT/NS (inhouse)  FEM/FVM  FNPT/NS  1phase/VOF  Inviscid/laminar 
LPT + WAMIT (inhouse)  BEM  LPT  Linearised  Inviscid 
Hybrid FNPT/SWENSE (inhouse)  HOS/FVM  FNPT/SWENSE  1phase/VOF  Inviscid/laminar 
OpenFOAM (overset)  FVM  NS  VOF  Laminar 
STARCCM +  FVM  NS  VOF  RANS (SST) 
WECSim 1  BEM  LPT  Linearised  Inviscid 
Nonlinear Froude–Krylov  Analytical  LPT  Linearised  Inviscid 
OpenFOAM (waves2Foam)  FVM  NS  VOF  Laminar 
WECSim 2  BEM  LPT  Linearised  Inviscid 
Using the hybrid Eulerian–Lagrangian particleincell (PIC) method, this model solves the incompressible NS equations for singlephase, freesurface flows. A Cartesian cutcellbased, twoway coupling algorithm is used to incorporate fluid–solid interaction (Chen et al., 2019d). A pistontype wave paddle is used to generate the waves with the displacement, based on firstorder wavemaker theory, derived by iteratively adjusting the theoretical focus location and amplitude. Wave absorption is achieved by way of an improved relaxation approach (Chen et al., 2019c). The computational domain is 6 m wide, 21 m long and 4 m high and has ∼32 million cells (edge length is 0.025 m) and ∼189 million particles. Dynamic time stepping is used with a Courant number, Co, of 0.5). No turbulence modelling is employed – that is, laminar flow is assumed. Computations were performed using 160 × 2.6 GHz cores (Chen et al., 2019b).
Based on the opensource, FVMbased, OpenFOAM (v4.1), this model solves the twophase, incompressible, Reynoldsaveraged NS (RANS) equations by way of volumeoffluid (VOF) interface capturing (Rusche, 2002). Dynamic mesh deformation accommodates the body motion. An impulse source method is used to generate the incident waves, with the source term determined by way of iterative calibration (Schmitt et al., 2019). A numerical beach implementation is used for wave absorption (Schmitt and Elsaesser, 2015). The computational domain is 7.825 m wide (utilising a symmetry plane), 28.75 m long and 6 m high (3 m of air phase) with ∼1 million cells (edge lengths 1.9–0.015 m). A fixed time step of 0.002 s is used. Laminar flow is assumed. 23 × 2.4 GHz cores were used for the computations (Windt et al., 2019).
This method computes the frequencydomain response of the body, including a linear spring for the mooring line, using LPT (Bingham, 2019; WAMIT Inc., 2019). A linear superposition of the Fourier coefficients, derived from the measured surface elevation signal at probe 5 in the empty tank test, is used to represent the incident waves. Only cases involving geometry 1 were simulated using the LPT method; a highorder Bsplines method, with 54 unknowns in total, represents the geometry exactly. One 2.5 GHz core was used for the computations.
This method decomposes the waveinteraction problem into: an incident wave part, which is solved using a highorder spectral (HOS) wave model based on FNPT, HOSNWT (Ducrozet et al., 2012); and a complementary part, which solves the twophase, spectral wave explicit Navier–Stokes equations (SWENSE) using a VOF method (implemented in OpenFOAM), foamStarSWENSE (Li et al., 2020). Communication between the two solvers is achieved using Grid2Grid (Choi et al., 2018). The entire physical tank is modelled; the wavemaker's motion is obtained using a timereversal procedure (Ducrozet et al., 2020), which reproduces the surface elevation at the structure position – that is, probe 5. The complementary part is solved in a domain encompassing the buoy, made up of hemispherical and cylindrical parts (radius = 5 m) with ∼0.6 M cells (edge length 0.01–0.25 m). Body motion is accommodated by way of meshdeformation. A fixed time step of 0.005 s is used. Laminar flow is assumed. 24 × 2.5 GHz cores are used. Only cases involving geometry 1 were simulated using the Hybrid FNPT/SWENSE method.
This method is based on LPT and utilises the opensource code WECSim (NREL and Sandia, 2014) to simulate the rigidbody timedomain responses by solving the Cummins’ equation with linear hydrodynamic coefficients obtained from the frequencydomain code WAMIT (WAMIT Inc., 2019). Incident waves are specified using a linear superposition of components derived using an FFT of the empty tank data at probe 5. Viscous drag coefficients are estimated by tuning the WECSim response to that using STARCCM + (described in 7); mooring forces are included by way of a coupling with opensource code MoorDyn (Hall, 2015); weakly nonlinear restoring and Froude–Krylov forcing terms are calculated by way of integration over a discretised body surface (2592 panels for geometry 1, 2048 panels for geometry 2) at each 0.01 s time step. Computations used 8 × 2.7 GHz cores (van Rij et al., 2019, 2020).
This model adds nonlinear kinematics and nonlinear Froude–Krylov force calculations to a LPTbased framework (Giorgi and Ringwood, 2018a, 2018b). A linear superposition of components, derived from the surface elevation at probe 5 (in the empty tank tests) and modified using Wheeler stretching, is used to approximate the incident waves, which are assumed to propagate linearly. No viscous drag correction is included. A fixed time step of 0.04 s is used. Computations were run on one 3.5 GHz core (Giorgi, 2019).
In this method, WECSim (NREL and Sandia, 2014) calculates the rigidbody timedomain responses by solving the Cummins’ equation with linear hydrodynamic coefficients, for each geometry, obtained using the frequencydomain, panelmethod, NEMOH (Babarit and Delhommeau, 2015) (∼2000 elements). Incident waves are derived from the instantaneous surface elevation (at probe 5) and impulse response functions calculated using BEMIO (NREL and Sandia, 2015). The mooring is included as a linear spring. Viscous drag coefficients are approximated using a steadystate, singlephase, NS solution (geometry 1) (Hughes et al., 2019, 2020) or by way of empirical data (geometry 2) (Hoerner, 1965). Nonlinear restoring or Froude–Krylov forces are not included. A 0.025 s time step is used. Computations used one 3.60 GHz core.
The CCPWSI Blind Test Series 2 is a blind comparative study of numerical WSI codes; only surface elevation measurements from an empty physical tank test, along with the test case description above, were available to the participants prior to submission. This information was considered to be sufficient for all participants to reproduce the incident waves and, thus, the cases with the structures in place. The remaining physical data were not released until after all participants had submitted their final results, and it is these ‘blind’ results that are reported here.
As mentioned in the previous two blind tests (Ransley et al., 2019, 2020), wellexecuted and wellunderstood physical measurement data is critical if a conclusive assessment of a numerical model's predictive capability is sought. In this case, the experiments were performed as part of the same physical test campaign used for the CCPWSI Blind Test Series 3 (Ransley et al., 2020); therefore, the same uncertainty due to random error is present (<2% relative standard deviation in the maximum displacements over five repeat tests). As in Series 3, the greatest source of uncertainty is in the mass properties of the two structures; due to practical limitations in the compound pendulum ‘swing test’ used (Hinrichsen, 2014), there is potential for significant systematic uncertainty in the moments of inertia supplied to the participants, and, consequently, discrepancies in the prediction of rotational motion – that is, pitch, might be anticipated. Despite this, as all participants have the same information, convergence of the numerical solutions, at least, is expected. Last, neither the mass or drag properties of the mooring line are known; participants were advised to assume the mooring system was ‘ideal’ – that is, massless, zerostiffness and offers no resistance to the fluid flow. As a consequence, any dynamic behaviour or drag on the mooring, that was present in the experiments, will not be captured by the numerical models.
As the participants had access to the empty tank data, and the situation is greatly simplified, convergence of the calculated surface elevation is expected (at least between similar methods). In general, the agreement between the numerical models and the experimental data is good, and there is less variation across the numerical solutions compared with the series 3 results (Ransley et al., 2020) (although the participants are different and some of those with the least good results in series 3 have not participated in series 2); however, some of the results again demonstrate that there can still be significant discrepancies, even in this ‘visible’ case, depending on the implementation strategy. When viewed in frequency space, the same observations can be made, however; the frequencydomain analysis appears to have filtered out the obvious discrepancies in the OpenFOAM (waves2Foam) result, which might only be highfrequency distortions, and has highlighted some curious issues underlying the OpenFOAM (sourceterm) result, which could be a consequence of the wave generation strategy used. In addition to this, it has been noticed that the quality of the reproduction varies with respect to the position interrogated and at other probe locations there is more spread in the results. Ignoring the linearised methods, which demonstrate significant discrepancies away from the target location (due to the nonlinear propagation of these waves not being described well by linear wave theory), the variations observed away from the target location highlight a clear issue with modern wave generation techniques; many of the methods that appear to perform the best use an iterative method to ‘tune’ their wave generation to achieve the desired surface elevation at the target location; however, if the result is not equally as good at other positions, can one be sure that the wave, including its underlying kinematics, has truly been reproduced as well as it appears to have been at the target location? Further, these methods tend to require the iterative procedure to be performed for every new case, which can increase the required computational resource considerably, particularly when using an already computational expensive method like CFD, and renders each simulation somewhat more difficult to reproduce by others. This issue is somewhat analogous to the ‘transfer functions’ used to control physical wavemakers which tend to be bespoke adjustments to the wavemaker displacements to achieve the desired result (typically only surface elevation is considered) at a particular position in the physical basin. This practice, in general, renders physical experiments as ‘facilityspecific’ and, analogously, iterative wave tuning renders numerical simulations as ‘methodspecific’ (albeit somewhat more transparent compared to the physical analogy). These observations, and the anticipated correlation between the quality of incident wave reproduction and that of the structural response, demonstrate a clear need for standardisation in numerical wave generation practices and also represent an additional complexity in judging the predictive capability of numerical models. There is no obvious trend in the quality of the reproductions as a function of wave steepness.
Figure 3 shows the predicted heave displacement of geometry 1's CoM when subject to the intermediate steepness wave, 2BT2, as well as the corresponding variance density spectra. In general, the predicted heave displacement is reasonably good across all model fidelities. Possibly because, in these cases, heave motion is dominated by inertia and restoring (rather than hydrodynamic) forces; however, there is a noticeable spread in the results (±10% of maximum heave) with the LPT model overpredicting and the WECSim and Froude–Krylov methods underpredicting the amplitude of heave displacement; the NS solvers typically perform slightly better, so there is some evidence to suggest the quality of the reproduction is a function of model fidelity (i.e., some nonlinear effects might be present). Despite this, as was the case in Series 3 (Ransley et al., 2020), the quality of the heave displacement prediction appears to correlate well with that of the surface elevation reproduction (for the nonlinear methods which model the wave propagation). In frequency space, the nonlinear/propagation models again have the same trends as in the surface elevation; the LPT method has an excellent main peak but significantly overestimates the higher frequency content; the Froude–Krylov method also has noticeable disparities at high frequencies (as well as underestimating the main peak); and there is considerable variation between the two WECSim results (particularly at high frequencies), perhaps suggesting that the estimate of the viscous drag and nonlinear forcing terms is important. In general, there is no obvious difference in the quality of the heave prediction with respect to the two geometries, although, for Geometry 2, the WECSim 2 implementation appears to have some issues with premature oscillation of the structure early in the time series. Again, there is no obvious trend in the quality of the reproduction as a function of wave steepness.
Figure 4 shows the predicted surge displacement of Geometry 1's CoM when subject to the intermediate steepness wave, 2BT2, along with the corresponding variance density spectra. As in series 3, there is considerably more spread in the surge results compared to heave, particularly after the main wave crest has passed and the structure displays significant ‘drift’ motion. Again, in general, the NS solvers and hybrid methods perform best and, out of these, those which reproduced the incident wave the best also appear to reproduce the surge motion the best; however, the two NS methods utilising overset meshing (STARCCM+ and OpenFOAM (overset)) do consistently predict premature surge motion that is not seen in the other methods or the physical data. Considering the linearised methods, the Froude–Krylov method does predict drift motion but has considerable discrepancies (possible due to assumed linear wave dispersion or exclusion of a viscous drag estimate); as expected, the pure LPT method does not predict any drift motion because no nonlinear effects are included; there is a clear difference between the two WECSim results: WECSim 2, which does not consider nonlinear forcing and has a more approximate strategy to estimate the viscous drag (Hughes et al., 2020), is missing the drift motion completely; whereas, WECSim 1, which includes weakly nonlinear restoring and Froude–Krylov forcing (based on the instantaneous body position and wave elevation) and utilises a NS solver to estimate the viscous drag coefficients (van Rij et al., 2020), gives a remarkably good prediction for the drift motion, comparable with the least good NS solvers. Despite the quality of the WECSim 1 prediction, all of the linearised methods used here neglect forces from the secondorder difference frequency in the spectrum (available in the NREL FAST software (Duarte et al., 2014)) as well as those from radiation and drag damping, which were derived in the frequency domain by Roald et al. (2013), for example, and shown to be important in predicting the mean drift of floating structures (Stansby et al., 2019). These observations suggest that the inclusion of nonlinear forcing terms, particularly those associated with the instantaneous body position (and, perhaps, more accurate estimates of viscous drag) are necessary to predict the surge motion in these focused wave cases. In frequency space, the variation in the prediction of lowfrequency drift motion is evident and the correlation with the incident wave prediction appears to be lost at low frequencies. Again, there is no obvious difference in the quality of the reproduction with respect to the two geometries; however, qualitatively, there is an apparent reduction in the predictive capability as a function of wave steepness (particularly for the linearised models but also generally).
As in the CCPWSI Blind Test Series 3 (Ransley et al., 2020), the mooring load in these cases is dominated by vertical motion – that is, heave. Consequently, the quality of the reproductions is similar to that of surface elevation and heave response (with some influence from surge motion) and there is no obvious difference in the quality of the reproduction as a function of either the wave steepness or the structure's geometric complexity. Curiously, all the models predict a greater peak in the variance density of the mooring load relative to the experiment, compared to that for the heave displacement.
As discussed in Ransley et al. (2019), raw timeseries analysis is valuable because the data can be considered in its entirety; however, this type of analysis makes trends in the data more difficult to interpret and, typically, only qualitative assessments can be made. Frequencydomain analysis offers an alternative visualisation of the timeseries data but has the same issues, when it comes to interpretation of trends. To make a quantitative analysis of the models predictive capabilities, and make progress towards a parametric understanding of the required model fidelity, the submitted data (and the physical data) need to be reduced to fewer, representative values. The challenge is, then, to identify appropriate measures of predictive capability that have no bias, towards a particular model, and are truly ‘representative’ – that is, independent of other factors that are considered unrelated to the ‘objective function’. This can be very difficult, in general, because the objective function in question typically depends on the application (and the end user's priorities), so the most appropriate measure of predictive capability is also likely to be a function of the specific use case under investigation.
In the CCPWSI Blind Test Series 3 (Ransley et al., 2020), a clear trend was observed between the normalised root meansquared (NRMS) error in the heave displacement and that of the surface elevation at the target location in the empty tank test. Using this measure, the capability of the models to predict heave motion was shown to correlate strongly with the quality of incident wave reproduction and the same trend is observed here (Figure 6). This suggested that all the NS solvers and hybrid methods have the same predictive capability for heave, given an equally good reproduction of the incident wave. This might have been anticipated because RMS tries to quantify the discrepancy in amplitude, and one might predict that a discrepancy in the incident wave amplitude corresponds to a discrepancy in vertical displacement, particularly when the heave is buoyancydriven – i.e., quasistatic; however, it was noted that, although the normalisation used provides the NRMS values with some independence from the specific case and the chosen interrogation window, RMS is a fairly crude measure of ‘similarity’ between two time series with strong sensitivity to phase discrepancies (not just amplitude) (Ransley et al., 2019, 2020). In general, this makes the value of an RMS analysis questionable, because for the majority of end users, although the amplitude of a displacement might be a key objective function, it is unlikely that a small phase shift (from an otherwise perfect solution) would be reason to penalise a particular method. In the Blind Test Series 3 (and here), it appears that, despite its limitations, an RMS analysis is a reasonable assessment of predictive capability in heave (probably because no significant phase discrepancies are observed and the heave motion, in the cases considered, is dominated by buoyancy); however, it is believed that the RMS is not a representative measure of predictive capability in either surge or pitch, and this is why similar trends have not been observed in these degrees of freedom. In response to this, an alternative, novel, quantitative analysis method is proposed in an attempt to provide measures of similarity, between time series, that are independent that is, isolate amplitude discrepancies from those in phase and frequency content.
Dynamic time warping (DTW) is a method for measuring the similarity between two data signals that vary in time. DTW was originally designed for automatic speech recognition (Sakoe and Chiba, 1978) and, in contrast to the linear alignment of signals by crosscorrelation, it is used to find the optimal global alignment between two time series by nonlinearly ‘warping’ the time axis of the signals until their dissimilarity is minimised. This process effectively removes, from the similarity measure, the influence of both ‘shifts’ in the time dimension – that is, phase discrepancies, and the ‘speeds’ of the two time series – that is, frequency discrepancies, offering, what is believed to be, an independent measure of the amplitude discrepancy between two time series. In addition, a ‘warping path’ is generated containing the information on how to translate, compress and expand the signals so that similar features are matched. DTW has been used in many fields, from analysis of electrocardiogram (ECG) measurements (Huang and Kinsner, 2002) to fault detection in wastewater treatment (Jun, 2011) and sewer flow monitoring (Dürrenmatt et al., 2013), but, despite this, to the authors’ knowledge, DTW has not been used to quantify the predictive capability of numerical models, particularly in the field of offshore and coastal engineering.
Figure 8 shows the Euclidean distances between the submitted numerical data and the physical measurements. The distances are calculated over an interrogation window from 35.3 to 50.3 s, from data interpolated onto a fixed sample frequency of 128 Hz, and are normalised by the standard deviation of the physical data over the same period (to achieve some independence with respect to the window size (Ransley et al., 2020)). Pitch results from the LPT method have been excluded due to an unresolved issue with the submitted data. Figure 8(a) shows that, when using this measure, the capability of the NS/hybrid codes, to predict the amplitudes of heave motion, correlates strongly with their ability to reproduce the incident wave amplitudes. This reinforces suggestions that the heave, in these cases, is dominated by inertia and restoring (rather than hydrodynamic) forces. Compared to NRMS (Figure 6), the use of Euclidean distance appears to separate the NS/hybrid methods into two distinct groups, and the trend in the data suggests a normalised Euclidean distance of zero is achievable, for heave displacement, provided perfect reproduction of the incident wave amplitudes. This suggests that the apparent limit in the capability to predict heave motion, found using NRMS (Ransley et al., 2020), is due to phase or frequency content discrepancies and the better performing group of NS methods have relatively higher discrepancies of this type compared to the poorer performing group. Figure 8(a) also shows that, when using Euclidean distance (as opposed to RMS), the capability of the linearised methods to predict heave motion is apparently improved, relative to the other methods, suggesting these methods suffer more from discrepancies in frequency and phase, than amplitude, when predicting heave motion. This observation demonstrates the benefit of being able to specify the wave elevation at the position of the structure precisely because the majority of the linearised predictions of the heave amplitudes are on par with the stronger group of NS/hybrid methods (and noticeably better than the weaker NS methods). Figure 8(b) shows the OpenFOAM (waves2Foam) method has a relatively poor prediction of the surge amplitudes, compared to the heave (and, therefore incident wave) amplitudes, and the OpenFOAM (sourceterm) method's prediction of surge amplitudes is stronger relative to heave. Other than this, in general, those NS/hybrid methods that predict the heave (and, therefore incident wave) amplitudes the best tend to predict the surge amplitudes the best (but the trend is far less pronounced compared to the heave–wave relationship). For the linearised methods, the spread in the predicted surge amplitudes is large: WECSim 2 and the nonlinear Froude–Krylov method have very poor predictions for the surge amplitudes of geometry 2 and geometry 1, respectively; but WECSim 1 has a remarkably low Euclidean distance for surge (even noticeably lower than that of the STARCCM+ method used to tune its viscous drag coefficients). Figure 8(c) shows that the spread in the predicted pitch angle amplitudes (plotted on a log scale) is the greatest of all the degrees of freedom considered with the majority of the linearised methods giving very poor predictions, particularly for geometry 2. The WECSim 1 implementation is, again, the exception with Euclidean distances for pitch comparable to the stronger NS/hybrid methods. These observations suggest that, for cases with unbroken waves interacting with simple floating systems, lower fidelity methods, like WECSim, can demonstrate predictive capabilities similar to highfidelity NS solvers, but only if improved estimates for the nonlinear terms are found beforehand. For NS/hybrid methods, the quality of the predicted pitch amplitudes tends to follow that of the heave (and, therefore incident wave) amplitudes, but, like surge, the trend is less pronounced compared with the relationship between heave prediction and incident wave reproduction. Curiously, considering its apparently superior performance in other degrees of freedom, the PIC method has abnormally poor predictions of the pitch amplitudes for cases with geometry 2; the OpenFOAM (source term) and STARCCM + methods also have greater pitch Euclidean distances for geometry 2, compared to geometry 1, but the OpenFOAM (waves2Foam) method has the opposite trend. Last, Figure 8(d) shows the Euclidean distances for pitch against those for surge; clearly (with the exception of WECSIM 1), the linearised methods are the weakest at predicting the amplitudes of pitch and surge motion; for the rest of the methods, there does not appear to be a clear trend between the Euclidean distances in pitch and surge, but, considering both degrees of freedom together, the PIC (geometry 1), OpenFOAM (source term) (geometry 1) and hybrid methods (along with the WECSim 1 implementation) appear superior to the rest of the NS solutions, which seem to be limited by a compromise between a strong performance in either pitch or surge (but not both).
As mentioned earlier, selecting the right numerical model for any particular application is considered to be a compromise between the model fidelity and the required computational effort to gain a solution. Figure 9 shows the normalised Euclidean distance for both heave and surge displacement against the CPU effort required to generate the solutions. CPU effort has been defined as the execution time of the numerical solver (in seconds), multiplied by both the number of cores used and their speed in GHz then divided by the simulated time in seconds. For methods requiring ‘preprocessing’ steps to complete the simulations – for example, tuning of the incident waves, the CPU effort required for these has been added when these steps must be repeated for every wave case (when the preprocessing only needs to be performed once per geometry for example, calculation of hydrodynamic coefficients, this effort has not been included). It should be noted that participants were not asked to minimise the CPU effort as part of this study. The WECSim 1 results have been plotted excluding the CPU effort required to estimate the viscous drag coefficients which, for the actual submitted results, required the STARCCM + results (for each case individually) to have been found prior. This technically means the WECSim 1 CPU effort is the greatest (six orders of magnitude higher than that plotted); however, this has been considered to be misleading because typically the viscous drag coefficients for each geometry would be estimated more economically (rather than for each incident wave case individually). Following the blind test, the use of a single set of drag coefficients (for each geometry) was found to have a fairly minimal effect on the results (van Rij et al., 2020).
As expected, the methods based on linear theory require significantly less CPU effort than the NS solvers (up to eight orders of magnitude less) and for heave displacement, at least, there is no significant improvement in the Euclidean distance to warrant the additional computing cost (for the cases considered here). Considering the NS solvers, the two hybrid methods do demonstrate significant savings in CPU effort, with the exception of the OpenFOAM (source term) method, which exploits the symmetry of the test cases and is therefore noticeably more efficient than the other pure NS solvers. As noted in series 1 (Ransley et al., 2019), there is a large range in the required CPU effort for the NS solvers. It is suspected that this is mainly due to the specific implementations – that is, mesh/domain design, applied by the individual operators (rather than the efficiency of the underlying code) and, although participants were not asked to minimise the CPU effort, this highlights another opportunity for bestpractice procedures to provide value.
The CCPWSI Blind Test Series 2 aims to audit the numerical methods presently in use, improve our understanding with respect to the appropriate model fidelity and inform the development of future numerical modelling standards, specifically in cases involving wave interactions with floating structures. To achieve these goals, volunteers are invited to simulate a series of test cases without prior access to the measurements from equivalent physical experiments. The Series 2 test cases consist of three focused waves, with a fixed crest height but variable steepness, incident on two separate, tautmoored, floating structures: a hemisphericalbottomed cylinder (Geometry 1) and a truncated cylinder with a moonpool (Geometry 2). To maximise the number of contributions to the test, no restrictions were placed on the accepted numerical methods. Further, participants were free to chose their preferred implementation strategy – for example, mesh design, parallel processing strategy and so on. Eleven different numerical methods are used in the test, ranging from those based on LPT to NS solvers, including meshbased and partialparticle methods, inhouse, open source and commercial codes as well as hybrid/coupled models.
Qualitative analysis of the submissions shows that, even with access to the physical data, when utilising a method that simulates the wave propagation, there can be significant differences in the quality of incident wave reproduction depending on the implementation strategy used; and the accuracy of the predicted structural response appears to correlate with that of the incident wave reproduction, particularly in the case of heave motion. In general, the NS solvers and hybrid methods do appear to have superior predictive capabilities compared to the linearised methods; however, heave is predicted reasonably well by all methods suggesting that, in these cases, the heave motion is not particularly sensitive to hydrodynamic forcing – that is, dominated by inertia and restoring forces. For both surge and pitch, there is considerable variation in the quality of the numerical predictions (even between similar models), again highlighting the need for bestpractice guidelines in numerical implementation (Ransley et al., 2019, 2020). In the cases considered, neither the wave steepness nor the structure's geometric complexity provide a clear distinction with respect to the required model fidelity; it is believed that the test cases, in future comparative studies, should span a step in the underlying physics – that is, unbroken–broken wave cases, to make this distinction. In addition to this, it is paramount that the number of variables, involved in future comparative studies, be minimised by enforcing standardised implementation strategies within each numerical method. It is believed this standardisation will reduce the variability in solutions from similar methods, enable identification of subtle differences between methods solving the same equations and provide further distinction between models solving different equations; however, in order that the true predictive capability of each method is represented fairly in the comparison, the enforced implementation strategies must resemble ‘best practice’ for each individual method and this is presently unknown (clearly evidenced by the variability observed throughout the CCPWSI Blind Test Workshops). To complicate things further, if one considers the common compromise between execution time and accuracy, the ‘best’ implementation of any particular model is also likely to be specific to the requirements of the end user.
An ancillary objective, arising from the CCPWSI Blind Test Workshops, is the development of robust, quantitative measures of predictive capability, without which a parametric understanding of the required model fidelity might never be achieved. Here, a novel, DTW approach is introduced in an attempt to provide such a measure. The DTW method provides a measure of similarity between two time series, the Euclidean distance, that is believed to be independent of discrepancies in both frequency and phase, making it more appropriate when quantifying amplitude discrepancies, compared to other more common analysis methods. DTW analysis also produces a separate indication of the phase and frequency discrepancies by way of the ‘warp path’ (the nonlinear warping of the time axis required to minimise the dissimilarity between the two signals being compared). The Euclidean distances between the submitted numerical data and the physical measurements provide quantitative confirmation of the qualitative observations. In general, the NS/hybrid methods perform better than the linearised methods, particularly when predicting surge and pitch motion, suggesting that at least these degrees of freedom represent a level of complexity that allows for differentiation between the predictive capabilities of different model fidelities; however, it should be noted that, there are no submissions using only FNPT so a gap is present in the model fidelities considered. As was the case in the CCPWSI Blind Test Series 3 (Ransley et al., 2020), when simulating the cases using NS/hybrid codes, the discrepancies in heave amplitude are strongly correlated to discrepancies in incident wave amplitude; for surge and pitch amplitudes the same trend is present but is far less well pronounced. This suggests that, for those methods that simulate the wave propagation, accurate reproduction of the incident waves is absolutely paramount, if an accurate prediction of body motion is sought. Despite this, at present, there is no consensus on the best strategy to reproduce the incident waves (evident from the number of methods used in this test) and there exists large variations in the quality of those methods employed, further emphasising the need for bestpractice guidelines and numerical modelling standards in WSI. For the methods based on LPT, in the cases considered here, the amplitude of heave motion is predicted reasonably well (in general, better than the weaker NS solvers); however, the amplitudes of surge and pitch motion are poorly predicted by all the LPT methods, except the WECSim 1 implementation which includes both weakly nonlinear restoring and Froude–Krylov forcing as well as viscous drag terms derived using a full NS solution to the same test cases. This demonstrates that, lower fidelity methods are capable of predicting these, apparently high fidelity, degrees of freedom but, if the considerable saving in the observed computational effort is to be realised, improved methods for determining viscous drag coefficients are required. This symbolises yet another area of WSI modelling that could benefit from the development of numerical modelling standards and bestpractice procedures. Finally, for the cases considered here (which, as mentioned above, all appear to represent the same physical complexity), the observations made suggest a relatively smooth tradeoff between accuracy and efficiency (at least for surge and pitch motion); however, for the NS solvers the range of computational resource utilised (for effectively the same outcome) is very large and this indicates yet another area in which best practice guidelines could add value to the WSI community.
Acknowledgements
The CCPWSI Working Group would like to acknowledge the participants of the CCPWSI Blind Test Series 2 for their contributions as well as the European Wave and Tidal Energy Conference (EWTEC) 2019 organisers for their support. This work is funded by the Engineering and Physical Sciences Research Council (EPSRC) through projects EP/M022382/1, EP/S000747/1 and EP/P026109/1. This research study was authored in part by the National Renewable Energy Laboratory, operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under contract number DEAC3608GO28308. Funding provided by U.S. Department of Energy Office of Energy Efficiency and Renewable Energy. The views expressed in this paper do not necessarily represent the views of the DOE or the U.S. Government. The U.S. Government retains and the publisher, by accepting the paper for publication, acknowledges that the U.S. Government retains a nonexclusive, paidup, irrevocable, worldwide license to publish or reproduce the published form of this study, or allow others to do so, for U.S. Government purposes. The full physical data set from the CCPWSI Blind Test Series 2 is available as a longstanding test case for future benchmarking and can be downloaded from CCPWSI (2020b).