Generation companies (GENCOs) are seeking for maximum economical profits in both energy and ancillary service markets. This paper proposes a methodology for obtaining the optimal bidding strategy for GENCOs in multiple electricity markets. Step-wise Supply function like bid curve with strategical bidding prices is used for the day-ahead multi-market auction. Probabilistic real time deployment of ancillary services is considered. Competition among GENCOs in such multi-market is modeled as a non-cooperative complete information game. A two-layer optimization problem is modeled, with the upper layer representing the GENCO's profit maximization subproblem and the lower layer representing the ISO's (Independent System Operator) simultaneous multi-market clearing subproblem. Illustrative examples show the advantages of considering probabilistic real time ancillary services deployment and simultaneously participating multi-market for a GENCO's profit seeking purpose.