We are concerned with the attainment of high performance in I/O on distributed, heterogeneous hardware. Our approach is to combine a program’s data retrieval and storage actions with operations executed on the resulting active I/O streams. Performance improvements are attained by exploitation of information about these operations and by runtime changes to their behavior and placement. In this fashion, active I/O can adjust to static system properties derived from the heterogeneous nature of resources and can respond to dynamic changes in system’s conditions, while reducing the total bandwidth needs and/or the end-to-end latencies of I/O actions.