• For Dose Escalation: Patients must have histologically or cytologically confirmed advanced or metastatic gastrointestinal (GI) cancers with Response Evaluation Criteria in Solid Tumors 1.1 (RECIST1.1) measurable disease who have progressed on at least one prior treatment for metastatic disease and for whom FOLFIRI is considered a reasonable treatment option. Patients with mismatch repair deficiency should have progressed on immunotherapy

• For Dose Expansion: Patients must have either:

‣ Colorectal cancer who have previously progressed on irinotecan and tolerated an irinotecan dose equal to or greater than the recommended phase 2 dose (RP2D). If they have mismatch repair deficiency they should have progressed on immunotherapy OR

⁃ Gastroesophageal cancer who have progressed on at least one first-line therapy for metastatic disease. If they have mismatch repair deficiency they should have progressed on immunotherapy

• For Dose Expansion: Patients be willing to undergo biopsies for research purposes only. The accessible tumor can be the primary or metastatic tumor site. Both research biopsies should be taken from the same tumor site

• Patients must have progressive disease on at least first-line therapy for metastatic disease. Previous treatment with irinotecan is allowed

• Age \>= 18 years. Because no dosing or adverse event data are currently available on the use of BAY 1895344 in combination with FOLFIRI in patients \<18 years of age, children are excluded from this study

• Eastern Cooperative Oncology Group (ECOG) performance status =\< 1

• Leukocytes \>= 3,000/mcL

• Absolute neutrophil count \>= 1,500/mcL

• Platelets \>= 100,000/mcL

• Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN)

• Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 x institutional ULN

• Glomerular filtration rate (GFR) \>= 60 mL/min/1.73 m\^2 using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation

• International normalization ratio (INR) =\< 1.5 × ULN unless participant is receiving anticoagulant therapy, in which case prothrombin time (PT) or activated partial thromboplastin time (aPTT) should be within expected therapeutic range of anticoagulants. If patient has a new diagnosis of venous thromboembolism (VTE), then patient should be appropriately anticoagulated with low molecular weight heparin (LMWH) or direct acting oral anticoagulants (DOACs) and be clinically stable for at least 1 week post treatment onset

• Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy, that does not interact with study therapy, with undetectable viral load within 6 months are eligible for this trial

• For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy that does not interact with study therapy, if indicated

• Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load and HCV therapy does not interact with study therapy

• Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression

• Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial

• Patients with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, patients should be class 2B or better

• The effects of BAY 1895344 on the developing human fetus are unknown. For this reason and because DNA-damage response inhibitors agents as well as other therapeutic agents used in this trial, 5-FU and irinotecan, are known to be teratogenic, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation and for 6 months after completion of BAY 1895344 administration

‣ Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 6 months after completion of study treatment administration

• Ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity (IDMC) who have a legally-authorized representative (LAR) and/or family member available will also be eligible