When an OPEN statement fails, a unit that was created for the OPEN needs
to be removed from the unit map. The code that tried to do this was
incorrect -- it needs to re-acquire the unit via LookUpForClose as a
CLOSE statement does. (The failure to do this completely was leaving a
zombie unit active that could break a later OPEN on the same unit
number.)
For testing purposes the implementations of the output APIs like
OutputInteger32 have been simply constructing descriptors and executing
the operation through the general DescriptorIO template. This patch
decouples those APIs from that mechanism so that programs using simple
"PRINT *" statements for output can link to a smaller portion of the I/O
runtime support library. (This is the only form of I/O accepted in GPU
device code by previous CUDA Fortran and Fortran OpenACC compilers.)
Supports the REDUCE() transformational intrinsic function of Fortran
(see F'2023 16.9.173) in a manner similar to the existing support for
SUM(), PRODUCT(), &c. There are APIs for total reductions to scalar
results, and APIs for partial reductions that reduce the rank of the
argument by one.
This implementation requires more functions than other reductions
because the various possible types of the user-supplied OPERATION=
function need to be elaborated.
Once the basic API in reduce.h has been approved, later patches will
implement lowering.
REDUCE() is primarily for completeness, not portability; only one other
Fortran compiler implements this F'2018 feature today, and only some
types work correctly with it.
This commit adds required files into the offload build closure,
which means adding RT_API_ATTRS and other markers.
The implementation does not work for CUDA yet, because of
std::variant,swap,reverse usage. These issues will be resolved
separately (e.g. by using libcudacxx header files).
This is a simplified implementation of std::optional that can be used
in the offload builds for the device code. The methods are properly
marked with RT_API_ATTRS so that the device compilation succedes.
Reviewers: klausler, jeanPerier
Reviewed By: jeanPerier
Pull Request: https://github.com/llvm/llvm-project/pull/85177
Finish plugging-in ASYNCHRONOUS IO in lowering (GetAsynchronousId was
not used yet).
Add a runtime implementation for GetAsynchronousId (only the signature
was defined). Always return zero since flang runtime "fakes"
asynchronous IO (data transfer are always complete, see
flang/docs/IORuntimeInternals.md).
Update all runtime integer argument and results for IDs to use the
AsynchronousId int alias for consistency.
In lowering, asynchronous attribute is added on the hlfir.declare of
ASYNCHRONOUS variable, but nothing else is done. This is OK given the
synchronous aspects of flang IO, but it would be safer to treat these
variable as volatile (prevent code motion of related store/loads) since
the asynchronous data change can also be done by C defined user
procedure (see 18.10.4 Asynchronous communication). Flang lowering
anyway does not give enough info for LLVM to do such code motions (the
variables that are passed in a call are not given the noescape
attribute, so LLVM will assume any later opaque call may modify the
related data and would not move load/stores of such variables
before/after calls even if it could from a pure Fortran point of view
without ASYNCHRONOUS).
The I/O runtime's API allows -1 to be passed for a unit number in a
READ, WRITE, or PRINT statement, where it gets replaced by 5 or 6 as
appropriate. This turns out to have been a bad idea, as it prevents the
I/O runtime from detecting and reporting a program's invalid attempt to
use -1 as an I/O unit number. So just pass 5 or 6 as appropriate.
Changes in libc++ during the development cycle for LLVM 17 lead to the FortranRuntime library depending on libc++.
Trying to build with Flang 17 that was built with clang++ 17 and libc++ 17 (on MinGW) leads to the following linker error:
ld.lld: error: undefined symbol: std::__1::__libcpp_verbose_abort(char const*, ...)
>>> referenced by libFortranRuntime.a(io-api.cpp.obj):(std::__1::__throw_bad_variant_access[abi:v170000]())
>>> referenced by libFortranRuntime.a(io-stmt.cpp.obj)
>>> referenced by libFortranRuntime.a(unit.cpp.obj)
That might be caused by std::get being called on a std::variant in common::visit.
std::__libcpp_verbose_abort is a weak symbol in libc++ that can be optionally replaced by an alternative definition in user code (see: [1])
Do that to avoid a dependency of the FortranRuntime on libc++.
[1]: https://libcxx.llvm.org/UsingLibcxx.html#overriding-the-default-termination-handler
See also: https://github.com/msys2/MINGW-packages/pull/18002#issuecomment-1694412640
Differential Revision: https://reviews.llvm.org/D158957
This patch enables the Fortran runtime support library to be
built without native 128-bit integer support in the C++ compiler.
Experimental: do not merge yet.
Differential Revision: https://reviews.llvm.org/D154660
It is an error to open a new unit with OPEN(NEWUNIT=) and have
neither a file name nor a scratch status. Catch it, and report a
new error code.
Differential Revision: https://reviews.llvm.org/D155967
OPEN statements can be used to change some, but not all, attributes
of units that have already been opened. The I/O runtime library
wasn't allowing ENCODING= to be changed. Every other Fortran compiler
permits this usage, and it's safe and useful, so allow it.
(Otherwise there's no good way to ensure that the preconnected
unit 6 is in UTF-8 mode.)
Differential Revision: https://reviews.llvm.org/D154379
12.10.2.17 defines that a INQUIRE statement's NEXTREC=n output value
for a unit that is not connected for direct access becomes undefined,
but the current I/O runtime can fail in a confusing manner by trying
to return uninitialized stack garbage.
Reported on Slack by Tarun Prabhu as an intermittent failure in
the gfortran regression test inquire_pre.f90.
Differential Revision: https://reviews.llvm.org/D152295
A fairly recent introduction of runtime I/O APIs called OutputDerivedType()
and InputDerivedType() didn't cover NAMELIST I/O's need to access
non-type-bound generic interfaces for user-defined derived type I/O
when those generic interfaces are defined in some scope other than the
one that defines the derived type.
The patch adds a new data structure shared between lowering
and the runtime that can represent all of the cases that can
arise with non-type-bound defined I/O. It can represent
scopes in which non-type-bound defined I/O generic interfaces
are inaccessible, too, due to IMPORT statements.
The data structure is now an operand to OutputDerivedType() and
InputDerivedType() as well as a data member in the NamelistGroup
structure.
Differential Revision: https://reviews.llvm.org/D148257
The present I/O infrastructure for user-defined derived type I/O
subroutines works fine for type-bound I/O generic bindings. It also works
for explicit INTERFACE blocks and GENERIC statements that define
UDDIO subroutines in the same scope as the definition of the derived type,
so long as the specific procedures in those bindings are module procedures
or external procedures.
For non-type-bound UDDTIO specific procedures that are dummy procedures,
thunks of inner procedures, or procedure pointers, or that are defined with
interfaces or GENERIC outside the scope of the definition of the derived
type, a new runtime I/O API is needed so that lowering can generate
a call that supplies the appropriate procedure as well as the defined
type instance.
This patch specifies and implements this new runtime API and provides
utility routines for lowering to use to determine whether it should be
called for any particular OutputItem or InputItem in the parse tree.
Differential Revision: https://reviews.llvm.org/D146571
A handful of I/O statements (OPEN, CLOSE, positioning) are not allowed
on units during child I/O; catch violations and report errors.
Also finesse error handling during FORMAT runtime parsing of DT
derived type edit descriptors, and ensure that formatted child
I/O is nonadvancing.
Differential Revision: https://reviews.llvm.org/D145751
Add the remaining pieces to support IO for noncontigous formats.
This is done by passing an array descriptor to IO calls. Scalar
formats continue to pass string and length arguments. IO calls
with formats are modified to place the new format descriptor
argument directly after the original string and length arguments.
A character array can be used as a format in an I/O data transfer
statement, with the interpretation that its elements are concatenated
in element order to constitute the format.
Support in the runtime with an extra optional descriptor argument
to six I/O API calls; support in semantics by removing an earlier
check for a simply contiguous array presented as a format.
Some work needs to be done in lowering to pass a character array
descriptor to the I/O runtime API when present
Differential Revision: https://reviews.llvm.org/D132167
When an I/O statement contains a function call that attempts
to perform I/O on the same unit, detect the recursive I/O
and terminate with a useful message rather than deadlocking in
the threading library.
Differential Revision: https://reviews.llvm.org/D131097
Some I/O control statements are no-ops when attempted on a bad or
unconnected UNIT=, but the standard says that FLUSH is an error
in that case.
Differential Revision: https://reviews.llvm.org/D128392
First, ExternalFileUnit::SetPosition was being used both as a utility
within the class' member functions as well as an API from I/O statement
processing. Make it private, and add APIs for SetStreamPos and SetDirectRec.
Second, ensure that SetStreamPos for POS= positioning in a stream
doesn't leave the current record number and endfile record number
in an arbitrary state. In stream I/O they are used only to manage
end-of-file detection, and shouldn't produce false positive results
from IsAtEnd() after repositioning.
Differential Revision: https://reviews.llvm.org/D128388
Fortran does have negative unit numbers -- they show up in child I/O
subroutines for defined I/O and for OPEN(NEWUNIT=) -- but the runtime
needs to catch the cases where a negative unit number that wasn't
generated by the runtime is passed in for OPEN or for an I/O statement
that would ordinarily create an anonymous "fort.NNN" file for a
hitherto unseen unit.
Differential Revision: https://reviews.llvm.org/D127788
When an unconnected unit number is used in a BACKSPACE statement
with ERR=, IOSTAT=, &/or IOMSG= control specifiers, don't crash,
but let the program deal with the error.
Differential Revision: https://reviews.llvm.org/D127782
When an I/O statement is known to be in a recoverable error state,
it shouldn't cause a crash later in execution because it's not in
an expected non-erroneous processing state. Add checking for the
ErroneousIoStatementState variant on paths that might otherwise
lead to runtime crashes.
Differential Revision: https://reviews.llvm.org/D127423
Track pending "asynchronous" I/O operation IDs so that WAIT statements can
report errors about bad ID numbers.
Lowering will need to extended to call GetAsynchronousId() for a READ or
WRITE statement with ID=n.
Differential Revision: https://reviews.llvm.org/D127421
Diagnose OPEN(FILE=f) when f is already connected by the same name to
a distinct external I/O unit.
Differential Revision: https://reviews.llvm.org/D127035
When an external I/O statement is in a recoverable error
state before any data transfers take place (for example,
an unformatted transfer with ERR=/IOSTAT=/IOMSG= attempted on
a formatted unit), ensure that the unit's mutex is still
released at the end of the statement.
Differential Revision: https://reviews.llvm.org/D127032
Whether a unit number in an inquire-by-unit statement is valid or not,
it should be the value to which the NUMBER= variable is set, not -1.
-1 should be returned to NUMBER= only for an inquire-by-file statement
when the FILE= is not connected to any unit.
Differential Revision: https://reviews.llvm.org/D126145
Now that the requirements and implementation of asynchronous I/O are
better understood, adjust their I/O runtime APIs. In particular:
1) Remove the BeginAsynchronousOutput/Input APIs; they're not needed,
since any data transfer statement might have ASYNCHRONOUS= and
(if ASYNCHRONOUS='YES') ID= control list specifiers that need to
at least be checked.
2) Add implementations for BeginWait(All) to check for the error
case of a bad unit number and nonzero ID=.
3) Rearrange and comment SetAsynchronous so that it's clear that
it can be called for READ/WRITE as well as for OPEN.
The implementation remains completely synchronous, but should be conforming.
Where opportunities make sense for true asynchronous implementations of
some big block transfers without SIZE= in the future, we'll need to add
a GetAsynchronousId API to capture ID= on a READ or WRITE; add sourceFile
and sourceLine arguments to BeginWait(All) for good error reporting;
track pending operations in unit.h; and add code to force synchronization
to non-asynchronous I/O operations.
Lowering should call SetAsynchronous when ASYNCHRONOUS= appears as
a control list specifier. It should also set ID=x variables to 0
until such time as we support asynchronous operations, if ever.
This patch only removes the removed APIs from lowering.
Differential Revision: https://reviews.llvm.org/D126143
When formatted CHARACTER input runs into the end of an input record,
the runtime usually fills the remainder of the variable with spaces,
but this should be conditional, and not done when PAD='NO'.
And while here, add some better comments to two members of connection.h
to make their non-obvious relationship more clear.
Differential Revision: https://reviews.llvm.org/D125053
Unit numbers must fit on a default integer. It is however possible that
the user provides the unit number in UNIT with a wider integer type.
In such case, lowering was previously silently narrowing
the value and passing the result to the BeginXXX runtime entry points.
Cases where the conversion caused overflow were not reported/caught.
Most existing compilers catch these errors and raise an IO error.
Add a CheckUnitNumberInRange runtime API to do the same in f18.
This runtime API has its own error management interface (i.e., does not
use GetIoMsg, EndIo, and EnableHandlers) because the usual error
management requires BeginXXX to be called to set up the error
management. But in this case, the BeginXXX cannot be called since
the bad unit number that would be provided to it overflew (and in the worst
case scenario, the narrowed value could point to a different valid unit
already in use). Hence I decided to make an API that must be called
before the BeginXXX and should trigger the whole BeginXXX/.../EndIoStatement
to be skipped in case the unit number is too big and the user enabled
error recovery.
Note that CheckUnitNumberInRange accepts negative numbers (as long as
they can fit on a default integer), because unit numbers may be negative
if they were created by NEWUNIT.
Differential Revision: https://reviews.llvm.org/D123157
Some refactoring and related fixes for more accurate
user program error recovery in the I/O runtime, especially
for error recovery with IOMSG= character values.
1) Move any work in an EndIoStatement() implementation
that may raise an error into a new CompleteOperation()
member function. This allows error handling APIs like
GetIoMsg() to complete a pending I/O statement and harvest
any errors that may result.
2) Move the pending error code from ErroneousIoStatementState
to a new pendingError_ data member in IoErrorHandler.
This allows IoErrorHandler::InError() to return a correct
result when there is a pending error that will be recovered
from so that I/O list data transfers don't crash in the meantime.
3) Don't create and leak a unit for a failed OPEN(NEWUNIT=n)
with error recovery, and don't modify 'n'. (Depends on
changes to API call ordering in lowering, in a separate patch;
code was added to ensure that OPEN statement control list
specifiers, e.g. SetFile(), must be passed before GetNewUnit().)
4) Fix the code that calls a form of strerror to fill an
IOMSG= variable so that it actually works for Fortran's
character type: blank fill with no null or newline termination.
Differential Revision: https://reviews.llvm.org/D122036
Where possible, I added additional information to the messages to help
programmers figure out what went wrong. I also removed all uses of the word
"bad" from the messages since (to me) that implies a moral judgement rather
than a programming error. I replaced it with either "invalid" or "unsupported"
where appropriate.
Differential Revision: https://reviews.llvm.org/D121493
The runtime crashes on several fundamental I/O data transfer statement
control list errors, like list I/O on a direct-access unit, or
input from a write-only unit, &c. These errors should not be fatal
when ERR= or IOSTAT= are present.
This patch creates a new ErroneousIoStatementState class and
uses it for the state of an I/O statement that is doomed to fail
from these errors. If there is no ERR= label or IOSTAT= variable,
the error will be raised at the end of the statement. Data transfer
operations along the way will be no-op failures.
Differential Revision: https://reviews.llvm.org/D120745
Add a header-only implementation of Briggs & Torczon's fast small
integer set data structure to flang/include/flang/Common, and use
it in the runtime to manage a pool of Fortran unit numbers with
recycling. This replaces the bit set previously used for that
purpose. The set is initialized on demand with the negations of
all the NEWUNIT= unit numbers that can be returned to any kind
of integer variable.
For programs that require more concurrently open NEWUNIT= unit
numbers than the pool can hold, they are now allocated with a
non-recycling counter. This allows as many open units as the
operating system provides.
Many of the top-line comments in flang/unittests/Runtime had the
wrong path name. I noticed this while adding a unit test for the
fast integer set data structure, and cleaned them up.
Differential Revision: https://reviews.llvm.org/D120685
Corrects the runtime implementation of I/O on files with
the access mode ACCESS='STREAM'. This is a collection
of edge-case tweaks to ensure that the distinctions between
stream and direct/sequential files, unformatted or formatted,
are respected where appropriate.
Moves NextInField() from io-stmt.h to io-stmt.cpp --
it was getting too big to keep in a header.
This patch exposed a problem with the I/O runtime
on Windows and it was reverted. This version also
fixes that problem; files are now opened on Windows
in binary mode to prevent inadvertent insertions of
carriage returns before line feeds, and those line
endings (CR+LF) are now explicitly generated.
Differential Revision: https://reviews.llvm.org/D119015
When a mode flag is modified (e.g., BLANK='ZERO') in an I/O data transfer
statement, ensure that the right set of mode flags is modified.
There's one set of mode flags that are captured by an OPEN
statement and maintained in the connection, and another that
is maintained in an I/O statement state record for local mutability.
Some I/O API routines were unconditionally modifying the persistent
set of flags.
Differential Revision: https://reviews.llvm.org/D118835
Corrects the runtime implementation of I/O on files with
the access mode ACCESS='STREAM'. This is a collection
of edge-case tweaks to ensure that the distinctions between
stream and direct/sequential files, unformatted or formatted,
are respected where appropriate.
Moves NextInField() from io-stmt.h to io-stmt.cpp --
it was getting too big to keep in a header.
Differential Revision: https://reviews.llvm.org/D118834
Use a bit-set to manage runtime-generated I/O unit numbers, recycle
them after they're closed, and use a range of values that fits in
a minimal-sized integer.
Differential Revision: https://reviews.llvm.org/D118651
In user-defined derived type I/O to an external unit, don't
omit the format string from the constructor of ChildFormattedIoStatement.
And include any user IOMSG text in the crash message of the
parent, if it doesn't catch errors.
Differential Revision: https://reviews.llvm.org/D117903
The unit number passed to a FLUSH statement is not required to
be a valid open unit; nothing happens (esp. not the creation of
an empty fort.n file) in this case.
Differential Revision: https://reviews.llvm.org/D117819
