1.0.0rc1 (July 6, 2020)

This is the first 1.0 release candidate.

API Changes

• Added solarforecastarbiter.reference_forecasts.utils.check_persistence_compatibility() to check if a pair of Forecast and Observation objects are compatible for making a persistence forecast (GH478)
• Added solarforecastarbiter.validation.tasks.validate_daily_defaults() to validate all observations using the immediate validation function along with stale and interpolated flags (GH482) (GH453)
• Added solarforecastarbiter.validation.tasks.apply_immediate_validation(), solarforecastarbiter.validation.tasks.apply_daily_validation(), and solarforecastarbiter.validation.tasks.apply_validation() convience functions (GH482)
• The dailyvalidation command of the CLI has been renamed to validate and a new --only-missing/--not-only-missing flag may be used to run validation only for periods that have not been previously validated (GH484)
• Added solarforecastarbiter.api.APISession.get_observation_values_not_flagged() to find days where an observation is not flagged with a given flag (GH484)
• Removed validation.tasks.immediate_observation_validation, validation.tasks.daily_single_observation_validation, and validation.tasks.daily_observation_validation in favor of solarforecastarbiter.validation.tasks.fetch_and_validate_observation() and solarforecastarbiter.validation.tasks.fetch_and_validate_all_observations() (GH484)
• Added support for probabilistic persistence forecasts with solarforecastarbiter.reference_forecasts.persistence.persistence_probabilistic() and solarforecastarbiter.reference_forecasts.persistence.persistence_probabilistic_timeofday() (GH195) (GH434)
• solarforecastarbiter.reports.figures.plotly_figures.timeseries_plots() now returns a third element to represent the probability vs. time plot if needed, otherwise this element is None. (GH467)
• Added capability to import reference data from the EIA Open Data API in the solarforecastarbiter.io.reference_observations.eia module. (GH431) (GH485)
• Added solarforecastarbiter.datamodel.ConstantCost, solarforecastarbiter.datamodel.TimeOfDayCost, solarforecastarbiter.datamodel.DatetimeCost, solarforecastarbiter.datamodel.ErrorBandCost, and solarforecastarbiter.datamodel.Cost and added solarforecastarbiter.metrics.deterministic.cost(), and added costs kwarg to solarforecastarbiter.metrics.preprocessing.process_forecast_observations() to support calculating deterministic cost metrics (GH487)
• Added a cost parameter and removed cost_per_unit_error from solarforecastarbiter.datamodel.ForecastObservation, solarforecastarbiter.datamodel.ForecastAggregate, and solarforecastarbiter.datamodel.ProcessedForecastObservation and added the costs parameter to solarforecastarbiter.datamodel.ReportParameters to support calculating cost metrics in reports (GH487)
• Removed solarforecastarbiter.metrics.valuation (GH487)
• Add solarforecastarbiter.reference_forecasts.make_latest_probabilistic_persistence_forecasts() to generate reference probabilistic persistence forecasts (GH490)
• solarforecastarbiter.io.api.get_probabilistic_forecast_values() now returns a DataFrame with string column names. It previously returned a DataFrame with floats for column names. For example, if the columns were previously described by Float64Index([25.0, 50.0, 75.0], dtype='float64') they are now described by Index(['25.0', '50.0', '75.0'], dtype='object'). (GH496)
• solarforecastarbiter.io.utils.serialize_timeseries() and solarforecastarbiter.io.utils.deserialize_timeseries() can now handle DataFrames in addition to Series. (GH495, GH496)
• solarforecastarbiter.reference_forecasts.utils.get_data_start_end() now requires an issue_time argument (GH497)
• Added a forecast_fill_method parameter to solarforecastarbiter.datamodel.ReportParameters (GH423) (GH493)
• Added solarforecastarbiter.metrics.preprocessing.apply_fill() that drops or fills missing or numpy.nan values with the last valid value or with a float. (GH423) (GH493)
• Added new arguments forecast_fill_method, start, and end to solarforecastarbiter.metrics.preprocessing.process_forecast_observations() (GH423) (GH493)

Enhancements

• Support parsing of Site climate zones from the API (GH481)
• Reapply validation to Observation data when fetched for a Report (GH376)
• Added the ‘DAILY VALIDATION APPLIED’ quality flag to ensure daily validation is applied to all observation values (GH377) (GH484)
• solarforecastarbiter.validation.tasks.fetch_and_validate_observation() and solarforecastarbiter.validation.tasks.fetch_and_validate_all_observations() can make use of solarforecastarbiter.api.APISession.get_observation_values_not_flagged() in order to only validate periods that have not yet had daily validation applied (GH377) (GH484)
• Reference probabilistic forecasts provided using the Persistence Ensemble (PeEn) method. (GH195) (GH434)
• Reports can be created for probabilistic forecasts. (GH266, GH467)
• Added net load reference observations from the US ISOs/RTOs: CAISO, ERCOT, ISO-NE, MISO, NYISO, PJM, and SPP. (GH431) (GH485)
• Implemented the calculation deterministic cost metrics in reports, including a constant cost factor, a time-of-day varying factor, a datetime varying factor, and a cost that depends on the magnitude of the error. (GH487) (GH501) (GH126) (GH492)
• Allow daily persistence forecasts to start at anytime instead of restricted to midnight UTC (GH497) (GH471)
• Added support for filling missing and numpy.nan data for forecasts. (GH423) (GH493)

Bug fixes

• Check if a Forecast and Observation are compatible for generating a persistence forecast before creating reference persistence forecasts (GH472) (GH478)
• When applying validation, only the ‘USER FLAGGED’ flag is preserved instead of all previous flags (GH484)
• Drop NaN before computing histogram when applying solarforecastarbiter.validation.validator.detect_clipping() (GH484)
• Fixed an issue where data for object pairs with forecast_type of event caused an AttributeError in the solarforecastarbiter.io.api module. (GH486, GH489)
• Fix ambiguity in forecast issue time of day (GH497)
• Fix data start/end for day of week net load persistence (GH497)
• Fix issue with inability to create ProbabilisticForecast and ProbabilisticForecastConstantValue from Forecast.from_dict. (GH502, GH503)
• Fix missing cells in validation, preprocessing, and metrics tables when an object did not contain the relevant data, and ensure correct ordering in PDF tables. (GH504, GH512, GH513)
• Fix bug where long forecast names were cut off when used as x axis labels on total metrics plots. (GH515)

Contributors

• Will Holmgren (@wholmgren)
• Leland Boeman (@lboeman)
• Cliff Hansen (@cwhanse)
• Tony Lorenzo (@alorenzo175)
• Justin Sharp (@MrWindAndSolar)
• Aidan Tuohy
• Adam Wigington (@awig)
• David Larson (@dplarson)