data_freshness()
1.00
0.00
Notes
Step 1 (freshness_details) ✓ Most recent data: 2026-07-13 16:51:16 (age: 1.0h, max allowed: 1d)
Actions class should be used to define the actions.
`brief: str | bool | None = None`
An optional brief description of the validation step that will be displayed in the reporting table. You can use the templating elements like `"{step}"` to insert the step number, or `"{auto}"` to include an automatically generated brief. If `True` the entire brief will be automatically generated. If `None` (the default) then there won't be a brief.
`active: bool | Callable = ``True`
A boolean value or callable that determines whether the validation step should be active. Using `False` will make the validation step inactive (still reporting its presence and keeping indexes for the steps unchanged). A callable can also be provided; it will receive the data table as its single argument and must return a boolean value. The callable is evaluated *before* any `pre=` processing. Inspection functions like has_columns() and has_rows() can be used here to conditionally activate a step based on properties of the target table.
`dimension: str | None = None`
An optional data quality dimension to categorize this validation step for health scoring. One of `"completeness"`, `"validity"`, `"uniqueness"`, `"consistency"`, `"timeliness"`, or `"volume"` (or any custom string). If `None` (the default), the dimension is inferred automatically from the assertion type. This label appears in the validation report and feeds the overall and per-dimension health scores.
## Returns
`Validate`
The [Validate](Validate.md#pointblank.Validate) object with the added validation step.
## How Timezones Affect Freshness Checks
Freshness validation involves comparing two times: the **data time** (the most recent timestamp in your column) and the **execution time** (when and where the validation runs). Timezone confusion typically arises because these two times may originate from different contexts.
Consider these common scenarios:
- your data timestamps are stored in UTC (common for databases), but you're running validation on your laptop in New York (Eastern Time)
- you develop and test validation locally, then deploy it to a cloud workflow that runs in UTC--suddenly your 'same' validation behaves differently
- your data comes from servers in multiple regions, each recording timestamps in their local timezone
The `timezone=` parameter exists to solve this problem by establishing a single, explicit timezone context for the freshness comparison. When you specify a timezone, Pointblank interprets both the data timestamps (if naive) and the execution time in that timezone, ensuring consistent behavior whether you run validation on your laptop or in a cloud workflow.
**Scenario 1: Data has timezone-aware datetimes**
``` python
# Your data column has values like: 2024-01-15 10:30:00+00:00 (UTC)
# Comparison is straightforward as both sides have explicit timezones
.data_freshness(column="updated_at", max_age="24 hours")
```
**Scenario 2: Data has naive datetimes (no timezone)**
``` python
# Your data column has values like: 2024-01-15 10:30:00 (no timezone)
# Specify the timezone the data was recorded in:
.data_freshness(column="updated_at", max_age="24 hours", timezone="America/New_York")
```
**Scenario 3: Ensuring consistent behavior across environments**
``` python
# Pin the timezone to ensure identical results whether running locally or in the cloud
.data_freshness(
column="updated_at",
max_age="24 hours",
timezone="UTC", # Explicit timezone removes environment dependence
)
```
## The `timezone=` Parameter
The `timezone=` parameter accepts several convenient formats, making it easy to specify timezones in whatever way is most natural for your use case. The following examples illustrate the three supported input styles.
**IANA Timezone Names** (recommended for regions with daylight saving time):
``` python
timezone="America/New_York" # Eastern Time (handles DST automatically)
timezone="Europe/London" # UK time
timezone="Asia/Tokyo" # Japan Standard Time
timezone="Australia/Sydney" # Australian Eastern Time
timezone="UTC" # Coordinated Universal Time
```
**Simple Hour Offsets** (quick and easy):
``` python
timezone="-7" # UTC-7 (e.g., Mountain Standard Time)
timezone="+5" # UTC+5 (e.g., Pakistan Standard Time)
timezone="0" # UTC
timezone="-12" # UTC-12
```
**ISO 8601 Offset Format** (precise, including fractional hours):
``` python
timezone="-07:00" # UTC-7
timezone="+05:30" # UTC+5:30 (e.g., India Standard Time)
timezone="+00:00" # UTC
timezone="-09:30" # UTC-9:30
```
When a timezone is specified:
- naive datetime values in the column are assumed to be in this timezone.
- the reference time (if naive) is assumed to be in this timezone.
- the validation report will show times in this timezone.
When `None` (default):
- if your column has timezone-aware datetimes, those timezones are used
- if your column has naive datetimes, they're treated as UTC
- the current time reference uses UTC
Note that IANA timezone names are preferred when daylight saving time transitions matter, as they automatically handle the offset changes. Fixed offsets like `"-7"` or `"-07:00"` do not account for DST.
## Recommendations For Working With Timestamps
When working with datetime data, storing timestamps in UTC in your databases is strongly recommended since it provides a consistent reference point regardless of where your data originates or where it's consumed. Using timezone-aware datetimes whenever possible helps avoid ambiguity--when a datetime has an explicit timezone, there's no guessing about what time it actually represents.
If you're working with naive datetimes (which lack timezone information), always specify the `timezone=` parameter so Pointblank knows how to interpret those values. When providing `reference_time=` as a string, use ISO 8601 format with the timezone offset included (e.g., `"2024-01-15T10:30:00+00:00"`) to ensure unambiguous parsing. Finally, prefer IANA timezone names (like `"America/New_York"`) over fixed offsets (like `"-05:00"`) when daylight saving time transitions matter, since IANA names automatically handle the twice-yearly offset changes. To see all available IANA timezone names in Python, use `zoneinfo.available_timezones()` from the standard library's `zoneinfo` module.
## Examples
The simplest use of [data_freshness()](Validate.data_freshness.md#pointblank.Validate.data_freshness) requires just two arguments: the `column=` containing your timestamps and `max_age=` specifying how old the data can be. In this first example, we create sample data with an `"updated_at"` column containing timestamps from 1, 12, and 20 hours ago. By setting `max_age="24 hours"`, we're asserting that the most recent timestamp should be within 24 hours of the current time. Since the newest record is only 1 hour old, this validation passes.
``` python
import pointblank as pb
import polars as pl
from datetime import datetime, timedelta
# Create sample data with recent timestamps
recent_data = pl.DataFrame({
"id": [1, 2, 3],
"updated_at": [
datetime.now() - timedelta(hours=1),
datetime.now() - timedelta(hours=12),
datetime.now() - timedelta(hours=20),
]
})
validation = (
pb.Validate(data=recent_data)
.data_freshness(column="updated_at", max_age="24 hours")
.interrogate()
)
validation
```
| STEP | COLUMNS | VALUES | TBL | EVAL | UNITS | PASS | FAIL | W | E | C | EXT | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #4CA64C | 1 |
data_freshness()
|
|
✓ | 1 | 1 1.00 |
0 0.00 |
-- | -- | -- | -- | ||
|
Notes Step 1 (freshness_details) ✓ Most recent data: |
|||||||||||||
| STEP | COLUMNS | VALUES | TBL | EVAL | UNITS | PASS | FAIL | W | E | C | EXT | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #4CA64C66 | 1 |
data_freshness()
|
|
✓ | 1 | 0 0.00 |
1 1.00 |
-- | -- | -- | -- | ||
| #4CA64C | 2 |
data_freshness()
|
|
✓ | 1 | 1 1.00 |
0 0.00 |
-- | -- | -- | -- | ||
| #4CA64C | 3 |
data_freshness()
|
|
✓ | 1 | 1 1.00 |
0 0.00 |
-- | -- | -- | -- | ||
|
Notes Step 1 (freshness_details) ✗ Most recent data: Step 2 (freshness_details) ✓ Most recent data: Step 3 (freshness_details) ✓ Most recent data: |
|||||||||||||
| STEP | COLUMNS | VALUES | TBL | EVAL | UNITS | PASS | FAIL | W | E | C | EXT | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #4CA64C | 1 |
data_freshness()
|
DUuODAzNzUwNDYsOC45MTg5Nzk3NSBDNy40MjkyODA5NSw4LjkxODk3OTc1IDguNzUxNzE4MDcsMTAuMjQxNDE2OSA4Ljc1MTcxODA3LDExLjg2Njk0NzQgQzguNzUxNzE4MDcsMTMuNDkyNDc3OCA3LjQyOTI4MDk1LDE0LjgxNDkxNSA1LjgwMzc1MDQ2LDE0LjgxNDkxNSBaIiBpZD0iU2hhcGUiIGZpbGw9IiMwMDAwMDAiIGZpbGwtcnVsZT0ibm9uemVybyIgLz4KICAgICAgICAgICAgPHBhdGggZD0iTTEzLjk2MzgxODksOC42OTkzMzUgQzEzLjkzNjQ2MjEsOC43MDQzMDkyNSAxMy45MDkxMDU5LDguNzExNzY5NjggMTMuODg0MjM1OSw4LjcxOTIzMDc0IEMxMy43ODIyNzA0LDguNzM2NjM5NjcgMTMuNjg3NzY1NCw4Ljc3NjQzMTE1IDEzLjYwNTY5NTYsOC44Mzg2MDUxOCBMMTAuMjQzMzE1NiwxMS4zODUyNTk4IEMxMC4wNzY2ODg2LDExLjUwNDYzNDMgOS45NzcyMDk5MywxMS42OTg2MTgxIDkuOTc3MjA5OTMsMTEuOTAyNTQ5MSBDOS45NzcyMDk5MywxMi4xMDY0ODA3IDEwLjA3NjY4ODYsMTIuMzAwNDYzOSAxMC4yNDMzMTU2LDEyLjQxOTgzODMgTDEzLjYwNTY5NTYsMTQuOTY2NDkzIEMxMy44OTE2OTcsMTUuMTgwMzcyNSAxNC4yOTcwNzI5LDE1LjEyMzE3MjEgMTQuNTEwOTUxNywxNC44MzcxNzA3IEMxNC43MjQ4MzEzLDE0LjU1MTE2OTIgMTQuNjY3NjMwOSwxNC4xNDU3OTQgMTQuMzgxNjI5NCwxMy45MzE5MTQ1IEwxMi41MzEzMjU3LDEyLjUzOTIxMjcgTDIxLjg4MTI0OTUsMTIuNTM5MjEyNyBMMjEuODgxMjQ5NSwxMS4yNjU4ODU0IEwxMi41MzEzMjU3LDExLjI2NTg4NTQgTDE0LjM4MTYyOTQsOS44NzMxODM2NCBDMTQuNjM3Nzg3Miw5LjcxNjUwNDUzIDE0Ljc0OTcwMDYsOS40MDA2NjAxNCAxNC42NDc3MzUxLDkuMTE3MTQ1NTMgQzE0LjU0ODI1NjQsOC44MzM2MzE1NiAxNC4yNjIyNTUsOC42NTk1NDM1MiAxMy45NjM4MTg5LDguNjk5MzM1IFoiIGlkPSJhcnJvdyIgZmlsbD0iIzAwMDAwMCIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoMTUuOTI5MjMwLCAxMS44OTQ3MzcpIHJvdGF0ZSgtMTgwLjAwMDAwMCkgdHJhbnNsYXRlKC0xNS45MjkyMzAsIC0xMS44OTQ3MzcpICIgLz4KICAgICAgICA8L2c+CiAgICA8L2c+Cjwvc3ZnPg==" />
✓ | 1 | 1 1.00 |
0 0.00 |
-- | -- | -- | -- | |||
|
Notes Step 1 (tz_warning) ⚠️ Column has naive datetimes but reference time is timezone-aware. Naive datetimes are being treated as if they're in the reference timezone. Step 1 (freshness_details) ✓ Most recent data: |
|||||||||||||
| STEP | COLUMNS | VALUES | TBL | EVAL | UNITS | PASS | FAIL | W | E | C | EXT | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #4CA64C | 1 |
data_freshness()
|
|
✓ | 1 | 1 1.00 |
0 0.00 |
-- | -- | -- | -- | ||
|
Notes Step 1 (tz_warning) ⚠️ Column has naive datetimes but reference time is timezone-aware. Naive datetimes are being treated as if they're in the reference timezone. Step 1 (freshness_details) ✓ Most recent data: |
|||||||||||||